Image processing method and device and computer readable storage medium

Abstract

The embodiment of the invention provides an image processing method and device and a computer readable storage medium. The method comprises the following steps: generating a group of original image vectors based on an original image; generating a group of target message vectors based on the target message; processing the group of original image vectors and the group of target message vectors based on a multi-head self-attention algorithm to generate a residual image; and generating a target image based on the residual image and the original image, the target image including the target message.

Description

Technical Field

[0001] The present disclosure relates to the field of image processing, and in particular, to methods, devices, and computer-readable storage media for image processing. Background Art

[0002] With the rapid development of technologies such as the Internet, computers, and communications, information dissemination has become increasingly convenient. However, at the same time, problems such as information security and confidentiality are faced. Thus, data steganography technology has received extensive attention. This technology usually uses a seemingly normal digital carrier as a cover to hide information in it for transmission, so as to avoid attracting the attention of network interceptors. One branch of data steganography technology is image steganography. How to efficiently implement image steganography has become one of the research hotspots. Summary of the Invention

[0003] In view of the need to improve the prior art, embodiments of the present disclosure provide methods, devices, and computer-readable storage media for image processing.

[0004] In a first aspect, embodiments of the present disclosure provide a method for image processing, including: generating a set of original image vectors based on an original image; generating a set of target message vectors based on a target message; processing the set of original image vectors and the set of target message vectors based on the Multi-Headed Self-Attention (MHSA) algorithm to generate a residual image; and generating a target image based on the residual image and the original image, where the target image contains the target message.

[0005] In a second aspect, embodiments of the present disclosure provide a method for image processing, including: processing a target image to generate a set of target image vectors, where the target image contains a target message; processing the set of target image vectors based on the MHSA algorithm to generate a set of message reconstruction vectors; and generating a target message based on the set of message reconstruction vectors.

[0006] In a third aspect, embodiments of the present disclosure provide a device for image processing, including: an image vector unit configured to generate a set of original image vectors based on an original image; a message vector unit configured to generate a set of target message vectors based on a target message; a first image generation unit configured to process the set of original image vectors and the set of target message vectors based on the multi-head self-attention MHSA algorithm to generate a residual image; and a second image generation unit configured to generate a target image based on the residual image and the original image, where the target image contains the target message.

[0007] Fourth aspect, embodiments of the present disclosure provide an apparatus for image processing, including: an image processing unit configured to: process a target image to generate a set of target image vectors, where the target image contains a target message; a vector generation unit configured to: process the set of target image vectors based on the multi-head self-attention (MHSA) algorithm to generate a set of message reconstruction vectors; a message generation unit configured to: generate a target message based on the set of message reconstruction vectors.

[0008] Fifth aspect, embodiments of the present disclosure provide an apparatus for image processing, including: at least one processor; a memory communicatively coupled to the at least one processor, having executable code stored thereon, the executable code, when executed by the at least one processor, causes the at least one processor to execute the method provided in the first aspect or the second aspect above.

[0009] Sixth aspect, embodiments of the present disclosure provide a computer-readable storage medium storing executable code, the executable code, when executed, implements the method provided in the first aspect or the second aspect above.

[0010] Seventh aspect, embodiments of the present disclosure provide a computer program product including a computer program, the computer program, when executed by a processor, implements the method provided in the first aspect or the second aspect above. Description of the Drawings

[0011] By a more detailed description of embodiments of the present disclosure in conjunction with the drawings, the above and other objects, features, and advantages of the embodiments of the present disclosure will become more apparent, where, in each of the drawings, the same reference numerals generally represent the same elements.

[0012] Figure 1 is a schematic diagram of an example scenario where the technical solution of the present disclosure can be applied.

[0013] Figure 2 is Figure 1 a schematic block diagram of an example of an encoding device in

[0014] Figure 3 is Figure 1 a schematic block diagram of an example of a decoding device in

[0015] Figure 4 is a schematic flowchart of a method for image processing according to some embodiments.

[0016] Figure 5 is a schematic flowchart of a method for image processing according to some embodiments.

[0017] Figure 6A It is a schematic diagram of an example of the functional architecture of an encoding device for implementing image processing according to some embodiments.

[0018] Figures 6B to 6G It shows an exemplary process of performing image processing using Figure 6A each module thereof.

[0019] Figure 7A It is a schematic diagram of an example of the functional architecture of a decoding device for implementing image processing according to some embodiments.

[0020] Figures 7B to 7D It shows an exemplary process of performing image processing using Figure 7A each module thereof.

[0021] Figure 8 It shows an example of the comparison between the original image and the target image.

[0022] Figure 9 It is a schematic block diagram of a device for image processing according to some embodiments.

[0023] Figure 10 It is a schematic block diagram of a device for image processing according to some embodiments. Detailed implementation manners

[0024] Now, the subject matter described herein will be discussed with reference to various embodiments. It should be understood that discussing these embodiments is only to enable those skilled in the art to better understand and implement the subject matter described herein, and is not to limit the scope of protection, applicability, or examples set forth in the claims.

[0025] Image steganography technology (or it can also be understood as a kind of "digital watermarking technology") may refer to hiding various forms of information using an image as a carrier. For example, a message can be hidden (or referred to as encoded, embedded, etc.) in an image. Generally, the image used to hide the message is usually called a cover or camouflage image (cover image) (or called the original image in this article), the hidden message is usually called a secret message (or called the target message in this article), and the image obtained after hiding the message in the cover image can be called a hidden or stego image (or called the target image in this article). The hidden image (or called the target image) can include information of both the cover or camouflage image and the secret message.

[0026] The present disclosure provides a technical solution for image processing to efficiently hide a message in an image without degrading the image quality and also to easily recover the hidden message from the image. It will be described below in conjunction with specific embodiments.

[0027] Figure 1 is a schematic diagram of an example scenario where the technical solution of the present disclosure can be applied. In Figure 1 this example, the encoding device 110 can process the original image and the target message to obtain a target image, and the target image can contain the target message. In some cases, the target image can contain the target message in a visually invisible manner, or the target image can contain the target message in a certain degree of visible manner (such as a substantially invisible manner), which can depend on the specific implementation. When the target image contains the target message with a certain degree of visual visibility, the visibility degree can be preset, such as based on the specific application scenario, business requirements, and so on. The encoding device 100 can transmit the target image to the decoding device 120. The decoding device 120 can process the target image to obtain the target message therein.

[0028] The encoding device 110 and the decoding device 120 can be any one of various devices. For example, they can be a desktop computer, a laptop computer, a mobile device, a tablet device, and so on. The encoding device 110 and the decoding device 120 can communicate in a wired or wireless manner.

[0029] Figure 2 is Figure 1 a schematic block diagram of an example of the encoding device 110 in

[0030] In Figure 2 this example, the encoding device 110 can include one or more processors 111, a memory 112, an input interface 113, and an output interface 114, and these structures can be coupled together through a bus 115. However, it should be understood that Figure 2 this is only an example for illustration and does not limit the scope of the present disclosure. For example, in different application scenarios, the encoding device 110 may include more or fewer structures, which are not limited herein.

[0031] The memory 112 can be used to store various data related to the functions or operations of the encoding device 110 (such as the original image, the target image, the target message, various vectors, etc. mentioned herein), executable instructions, or executable code, etc. For example, the memory 112 can include but is not limited to random access memory (RAM), read-only memory (ROM), flash memory, programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), registers, hard disks, and so on.

[0032] The processor 111 can be used to execute or implement various functions or operations of the encoding device 110, such as the image processing operations described herein. For example, the processor 111 can execute the executable code or instructions stored in the memory 112, thereby implementing various processes described in connection with the various embodiments of the present disclosure. The processor 111 can include various applicable processors, such as, for example, a general-purpose processor (such as a central processing unit (CPU)), a dedicated processor (such as a digital signal processor, a graphics processing unit, an application-specific integrated circuit, etc.).

[0033] The input interface 113 can receive various forms of data or information, etc. For example, the input interface 113 can receive the original images, target messages, etc. mentioned herein. In some implementations, the input interface 113 can include a keyboard, a microphone, a touch screen display, a camera, etc. In some implementations, the input interface 113 can also include an interface for communicating with other entities based on wired or wireless communication standards.

[0034] The output interface 114 can output various forms of data or information, etc. For example, the output interface 114 can output the target images described herein, etc., for transmission to the decoding device 120. In some implementations, the output interface 114 can be implemented based on various applicable wired or wireless communication standards.

[0035] Figure 3 is Figure 1 a schematic block diagram of an example of the decoding device 120 in.

[0036] In Figure 3 the example of, the decoding device 120 can include one or more processors 121, a memory 122, an input interface 123, and an output interface 124, and these structures can be coupled together through a bus 125. However, it should be understood that Figure 3 is only an illustration and does not limit the scope of the present disclosure. For example, in different application scenarios, the decoding device 120 may include more or fewer structures, which are not limited herein.

[0037] The memory 122 can be used to store various data (such as the target images, target messages, various vectors, etc. mentioned herein), executable instructions, or executable code, etc. related to the functions or operations of the decoding device 120. For example, the memory 122 can include, but is not limited to, RAM, ROM, flash memory, PROM, EPROM, registers, hard disks, etc.

[0038] The processor 121 can be used to execute or implement various functions or operations of the decoding device 120, such as the image processing operations described herein. For example, the processor 121 can execute the executable code or instructions stored in the memory 122, thereby implementing the various processes described in connection with the various embodiments of the present disclosure. The processor 121 can include various applicable processors, such as a general-purpose processor (such as a CPU), a dedicated processor (such as a digital signal processor, a graphics processing unit, an application-specific integrated circuit, etc.).

[0039] The input interface 123 can receive various forms of data or information, etc. For example, the input interface 123 can receive a target image. In some implementations, the input interface 123 can be implemented based on various applicable wireless or wired communication standards.

[0040] The output interface 124 can output various forms of data or information, etc. For example, the output interface 124 can output the target message mentioned herein. In some implementations, the output interface 124 can include a display screen, a speaker, etc. In addition, the output interface 124 can also be implemented based on various applicable wireless or wired communication standards.

[0041] Figure 4 is a schematic flowchart of a method for image processing according to some embodiments. Figure 4 The method of Figure 1 can be executed by the encoding device 110 of

[0042] such as, for example, the processor 111 of the encoding device 110. Figure 4 As shown in

[0043] In step 402, a set of original image vectors can be generated based on the original image.

[0044] In step 404, a set of target message vectors can be generated based on the target message.

[0045] In step 406, a set of original image vectors and a set of target message vectors can be processed based on the MHSA algorithm to generate a residual image.

[0046] In an embodiment of the present disclosure, a set of original image vectors can be generated from an original image, and a set of target message vectors can be generated from a target message. Then, based on the MHSA algorithm, a set of original image vectors and a set of target message vectors are processed to generate a residual image. After that, based on the residual image and the original image, a target image containing the target message can be generated. By using a deep learning algorithm such as MHSA, the target message can be efficiently hidden in the original image, and the quality of the target image can be ensured. This is also beneficial for subsequently recovering the target message from the high-quality target image.

[0047] In some embodiments, the target message may have a binary format, that is, it can be represented as a bit sequence (for example, a sequence of 0s and / or 1s). The total number of bits of the target message may be D times the total number of pixels of the original image. D is greater than 0. D can be an integer or may not be an integer. This is not limited herein. In some cases, for the convenience of evaluating the performance of hiding messages, D may be a positive integer. For example, D can be 1, 2, 3, 4, 5, 6, 7, 8, etc. Assume that the size of the original image is H_im*W_im, that is, it has H_im*W_im pixels. H_im can represent the height of the original image, and W_im can represent the width of the original image. Then, H_im*W_im*D bits can be hidden in the original image. It can be seen that in the embodiments of the present disclosure, a large number of bits can be hidden in the original image, such as several times the total number of pixels. This can also be understood as the hiding density of the message being D bits per pixel. For example, if the original image has 256*256 pixels and the target message has 256*256 = 65536 bits, then D is 1, that is, 256*256 = 65536 bits can be hidden in the original image, thus achieving a hiding density of 1 bit per pixel. The above is an example with the multiple D being an integer. In some cases, the multiple D may not be an integer. For example, if the original image has 224*224 pixels and the target message has 25088 bits, then the multiple D can be 25088 / (224*224) = 0.5. Whether the multiple D is an integer greater than 0 or a non-integer, it can be seen that the embodiments of the present disclosure can achieve high-density bit hiding or steganography.

[0048] In some embodiments, the target message may have multiple message segments, and each message segment has the same number of bits. Specifically, the target message can be divided into multiple message segments. In this way, it can be beneficial to process the target message efficiently, especially for messages with a large number of bits. In addition, this approach is also beneficial for calculating the message hiding density. The number of bits in each message segment can be greater than 1, specifically, it can be greater than or equal to 2, such as 2, 4, 8, 16, etc. In some implementation manners, each message segment can be encoded to obtain a corresponding target message vector. In addition, in some implementation manners, each target message vector and each original image vector can have the same length (also referred to as dimension), which can be beneficial for subsequent operations, such as vector addition operations (which will be described in detail below).

[0049] In some cases, when encoding the message segment, it may involve adjusting the length or dimension so that the length of the obtained target message vector is equal to the length of the original image vector. For example, the original image vector is usually a high-dimensional vector, while the number of bits in the message segment is usually relatively small. In this case, the dimension can be increased when encoding the message segment so that the length of the obtained target message is equal to the length of the original image vector.

[0050] Regarding the number of message segments and the number of bits in each message segment, they can be set according to actual service requirements, application scenarios, etc., and this is not limited in this article. For example, the number of message segments can be defined with reference to the total number of pixels of the original image or the total number of pixels of the feature map obtained from the original image.

[0051] In some implementation manners, the total number of multiple message segments can be equal to the total number of pixels of the original image. In this case, the original image can actually be encoded into a feature map with the same height and width but an increased number of channels, and then a set of original image vectors can be obtained. The feature map can be understood as including a set of original image vectors, and the number of a set of original image vectors can be equal to the height * width of the feature map. In this case, the number of bits in the message segment can actually be equal to the multiple D, which is also more convenient for calculating the message hiding density.

[0052] In some implementation manners, the total number of multiple message segments can be less than or equal to the total number of pixels of the feature map. In this case, the size of the feature map obtained by encoding the original image can be different from the size of the original image. For example, the height and / or width of the feature map can be less than the height and / or width of the original image, but the number of channels can be greater than the number of channels of the original image. The feature map actually corresponds to a set of original image vectors, and it can also be understood that the feature map is the height * width original image vectors.

[0053] In some embodiments, the target image may present the same visual content as the original image. Of course, in other implementations, the target image may also present visual content that is not exactly the same as the original image. This depends on specific implementation methods, business requirements, etc., and is not limited herein.

[0054] In some embodiments, in step 402, a convolutional algorithm or a Multilayer Perceptron (MLP) algorithm may be used to encode the original image to obtain a set of original image vectors. This can be achieved through existing applicable methods, which are not limited herein. For example, a convolutional algorithm can be used to perform convolutional processing on the original image to obtain a feature map, and then a set of original image vectors can be obtained based on the feature map.

[0055] In some embodiments, in step 404, at least the MLP algorithm may be used to encode the target message to obtain a set of target message vectors. The bits in the target message are usually sequential, and the MLP algorithm is usually position-sensitive. Therefore, using at least the MLP algorithm to encode the target message, the resulting output can imply the sequentiality of the bits in the target message. In this way, the obtained set of target message vectors can more comprehensively reflect the characteristics of the target message, which is not only beneficial for hiding the target message into the original image but also beneficial for subsequently recovering the target message from the target image.

[0056] In some embodiments, only the MLP algorithm may be used to encode the target message to obtain a set of target message vectors. In some implementations, at least one MLP layer may be used to encode the target message to obtain a set of target message vectors. For example, if multiple MLP layers are used, the multiple MLP layers may be arranged in sequence. The first MLP layer may process based on the target message, each MLP layer after the first MLP layer may process based on the output of the previous MLP layer, and the last MLP layer may output a set of target message vectors.

[0057] In some embodiments, the MLP algorithm and the MHSA algorithm can be used to encode the target message to obtain a set of target message vectors, which can significantly improve the quality of the target message vectors. In this case, the MLP algorithm can be first used to process the target message, and then the MHSA algorithm can be used to process the output obtained by the MLP algorithm, so as to obtain a set of target message vectors. When the MLP algorithm encodes the target message, it usually focuses on the local distribution of bits in the target message. Using the MHSA algorithm to further process the output obtained by the MLP algorithm can globally optimize the result of message encoding. For example, when the target message includes multiple message segments, the MLP algorithm usually mainly focuses on the bit distribution in each message segment, and thus gives the encoding result based on the message segment. The MHSA algorithm can be a global operation, so the MHSA algorithm can perform global encoding on the output of the MLP algorithm, which can also be understood as the MHSA algorithm encoding all message segments again, so as to optimize message encoding and improve the quality of the obtained target message vectors. In some implementation manners, at least one MLP layer and at least one MHSA layer can be used to encode the target message to obtain a set of target message vectors. In this case, at least one MLP layer can be located before at least one MHSA layer. If multiple MLP layers and multiple MHSA layers are used, the multiple MLP layers can be arranged in sequence. After the last MLP layer, multiple MHSA layers can be arranged, and the multiple MHSA layers can also be arranged in sequence. In addition, for the embodiments using the MLP algorithm and the MHSA algorithm to encode the target message, the position information can also be considered. For example, in order to optimize the processing performance of the MHSA algorithm, the output of the MLP algorithm and the position information of each message segment can be input into the MHSA algorithm together. The position information can represent the order or position of the message segments, and is actually the order or position of the vectors corresponding to the message segments (i.e., the vectors obtained by processing through the MLP algorithm). Generally, the position information can include a set of position vectors, and each position vector can correspond to a message segment, that is, correspond to a message vector. In this way, the set of target message vectors obtained from the MHSA algorithm will itself contain position information.

[0058] After obtaining a set of original image vectors and a set of target message vectors, they can be processed to obtain a residual image. In some embodiments, a vector addition operation can be performed on a set of original image vectors and a set of target message vectors to obtain a set of input vectors. The MHSA algorithm can be used to process the set of input vectors to obtain a set of fusion vectors. Then, a residual image can be generated based on the set of fusion vectors.

[0059] In some embodiments, the number of a set of original image vectors may be equal to or unequal to the number of a set of target message vectors. For the convenience of efficient processing, the number of a set of target message vectors can generally be less than or equal to the number of a set of original image vectors. Whether the number of a set of target message vectors is less than or equal to the number of a set of original image vectors, a set of original images and a set of target message vectors can be added in sequence. If the number of a set of original image vectors is equal to the number of a set of target message vectors, a set of original image vectors and a set of target message vectors can be added respectively to obtain a set of input vectors. For example, the first original image vector in a set of original image vectors can be added to the first target message vector in a set of target message vectors to obtain the first input vector; the second original image vector can be added to the second target message vector to obtain the second input vector; and so on until the last input vector is obtained.

[0060] If the number of a set of target message vectors is less than the number of a set of original image vectors, a part of the original image vectors in a set of original image vectors can be added to a set of target message vectors to obtain a first part of input vectors. The number of this part of the original image vectors can be equal to the number of a set of target message vectors. In this case, a set of input vectors can include the first part of input vectors and a second part of input vectors. The remaining original image vectors in a set of original image vectors except for the part of the original image vectors used for vector addition can be used as the second part of input vectors.

[0061] Specifically, since the number of a set of target message vectors is less than the number of a set of original image vectors, a part of the original image vectors in a set of original image vectors can be selected to be added to a set of target message vectors respectively to obtain some input vectors. For example, the first original image vector can be added to the first target message vector to obtain the first input vector; the second original image vector can be added to the second target message vector to obtain the second input vector; and so on until all the target message vectors have undergone the vector addition operation. The remaining original image vectors can be directly used as input vectors without performing the vector addition operation.

[0062] As can be seen from the above description, when performing the vector addition operation, the addition can be performed respectively in the order of a set of original image vectors and a set of target message vectors. As mentioned above, a set of target message vectors obtained after encoding the target message can itself contain position information. In this way, the set of input vectors obtained actually also contains position information. Therefore, in some implementation manners, when processing a set of input vectors using the MHSA algorithm, it may not be necessary to additionally add the position information of a set of input vectors. This also helps to reduce the amount of computation. Of course, in some implementation manners, the position information of a set of input vectors and a set of input vectors can be input into the MHSA algorithm for processing together. The position information of a set of input vectors can include a set of position vectors, and each position vector can correspond to an input vector, which is used to represent the order or position of the input vector. This can also further improve the processing performance of the MHSA algorithm for a set of input vectors.

[0063] In addition, the MHSA algorithm generally mainly processes three factors, the Query (Q) factor (hereinafter abbreviated as the Q factor), the Key (K) factor (hereinafter abbreviated as the K factor), and the Value (V) factor (hereinafter abbreviated as the V factor). These three factors are abbreviated as these three factors are essentially three matrices. The MHSA algorithm can obtain an output by performing corresponding operations on these three factors.

[0064] In the embodiments of the present disclosure, the MHSA algorithm for processing a set of input vectors can be implemented by using one or more MHSA layers.

[0065] In some embodiments, the MHSA algorithm can be implemented by using a single MHSA layer. Specifically, a single MHSA layer can be used to process a set of input vectors to obtain a set of fused vectors.

[0066] In this case, in this single MHSA layer, the K factor and the V factor of this MHSA layer can be determined based on a set of input vectors. For example, different transformations (such as linear transformations) can be performed on a set of input vectors to respectively obtain the K factor and the V factor.

[0067] The Q factor of this MHSA layer can be determined based on a set of input vectors, a set of original image vectors, or a set of target message vectors. For example, an appropriate transformation (such as a linear transformation) can be performed on a set of input vectors to obtain the Q factor. For another example, an appropriate transformation (such as a linear transformation) can be performed on a set of original image vectors to obtain the Q factor. For yet another example, an appropriate transformation (such as a linear transformation) can be performed on a set of target message vectors to obtain the Q factor.

[0068] By determining the K factor, V factor, and Q factor of the MHSA layer in this way, the processing performance of the MHSA layer can be effectively improved.

[0069] In some embodiments, multiple MHSA layers can be used to implement the MHSA algorithm. Specifically, multiple MHSA layers can be used to process a set of input vectors to obtain a set of fused vectors. The multiple MHSA layers can be arranged sequentially. For example, the first MHSA layer among the multiple MHSA layers can process based on the above-mentioned set of input vectors, and each MHSA layer except the first MHSA layer can process based on the output of the previous MHSA layer of this MHSA layer, and the last MHSA layer can output a set of fused vectors.

[0070] By using multiple MHSA layers to obtain a set of fused vectors, the overall processing performance can be more effectively improved, the target message can be reliably hidden in the original image, and it is beneficial to improve the quality of the target image, which in turn helps to efficiently recover the target message from the target image subsequently.

[0071] In the case of using multiple MHSA layers, the respective K factor, V factor, and Q factor can be processed in each MHSA layer to obtain a set of output vectors. It can be understood that the set of output vectors obtained from the last MHSA layer can be used as the above-mentioned set of fused vectors.

[0072] In the first MHSA layer, based on a set of input vectors, the K factor and V factor of the first MHSA layer can be determined. For example, different transformations can be performed on a set of input vectors to respectively obtain the K factor and V factor of the first MHSA layer. In addition, based on a set of input vectors, a set of original image vectors, or a set of target message vectors, the Q factor of the first MHSA layer can be determined. For example, appropriate transformations can be performed on a set of input vectors, a set of original image vectors, or a set of target message vectors to obtain the Q factor of the first MHSA layer.

[0073] In any MHSA layer except the first MHSA layer, based on a set of output vectors and a set of target message vectors of the previous MHSA layer of this MHSA layer, the K factor and V factor of this MHSA layer can be determined; in addition, based on both a set of output vectors and a set of target message vectors of the previous MHSA layer of this MHSA layer, or based on a set of target message vectors, or based on a set of original image vectors, the Q factor of this MHSA layer can be determined.

[0074] By determining the K factor, V factor, and Q factor for different MHSA layers in different ways, the processing performance of the MHSA algorithm can be effectively improved.

[0075] In some embodiments, in any MHSA layer except the first MHSA layer, a vector addition operation may be performed on a set of output vectors of the previous MHSA layer of this MHSA layer and a set of target message vectors to obtain a set of vectors to be processed for this MHSA layer. For example, in a similar manner of adding a set of original image vectors and a set of target message vectors, a vector addition operation may be performed on a set of output vectors of the previous MHSA layer of this MHSA layer and a set of target message vectors.

[0076] Based on the set of vectors to be processed for this MHSA layer, the K factor and V factor of this MHSA layer may be determined. For example, different transformations may be performed on this set of vectors to be processed to respectively obtain the K factor and V factor of this MHSA layer.

[0077] Based on the set of vectors to be processed for this MHSA layer, a set of target message vectors, or a set of original image vectors, the Q factor of this MHSA layer may be determined. For example, appropriate transformations may be performed on the set of vectors to be processed for this MHSA layer, or on a set of target message vectors, or on a set of original image vectors to obtain the Q factor of this MHSA layer.

[0078] After obtaining a set of fused vectors based on the MHSA algorithm, a residual image may be generated based on this set of fused vectors. Any appropriate image reconstruction algorithm may be used to generate the residual image, which is not limited herein. Then, based on the residual image and the original image, a target image may be generated. In some embodiments, to improve the quality of the target image, the residual image may be added to the original image to obtain the target image. For example, the pixel values of the corresponding pixels of the residual image and the original image may be added to obtain the pixel values of the corresponding pixels of the target image.

[0079] The above describes from the encoding perspective how to obtain the target image containing the target message, that is, the hidden or steganographic image. Next, how to recover the target message from the target image will be described from the decoding perspective.

[0080] Figure 5 is a schematic flowchart of a method for image processing according to some embodiments. Figure 5 The method 500 may be executed by Figure 1 the decoding device 120 (e.g., the processor 121 of the decoding device 120).

[0081] As Figure 5 shown, in step 502, the target image may be processed to generate a set of target image vectors. The target image may contain the target message.

[0082] In step 504, a set of target image vectors can be processed based on the MHSA algorithm to generate a set of message reconstruction vectors.

[0083] In step 506, a target message can be generated based on a set of message reconstruction vectors.

[0084] In an embodiment of the present disclosure, a set of target image vectors generated from a target image are processed based on the MHSA algorithm to obtain a set of message reconstruction vectors, and then, based on this set of message reconstruction vectors, the target message contained in the target image can be recovered with high quality.

[0085] In some embodiments, a convolutional algorithm or an MLP algorithm can be used to encode the target image to obtain a set of target image vectors.

[0086] In some embodiments, in step 504, the MHSA algorithm can be implemented using one or more MHSA layers.

[0087] In some implementation manners, a single MHSA layer can be used to process a set of target image vectors to obtain a set of message reconstruction vectors.

[0088] In this case, in this single MHSA layer, a K factor, a V factor, and a Q factor can be determined based on a set of target image vectors, and then, the K factor, the V factor, and the Q factor can be processed to obtain a set of message reconstruction vectors.

[0089] For example, different transformations (such as linear transformation) can be performed on a set of target image vectors to respectively obtain the K factor and the V factor of this single MHSA layer.

[0090] The Q factor of a single MHSA layer can be obtained in various ways. In some embodiments, appropriate transformations (such as linear transformation) can be performed on a set of target image vectors to obtain the Q factor of the single MHSA layer. In some embodiments, a query learner can be used to process a set of target image vectors to obtain the Q factor of this single MHSA layer. The query learner can adopt an MLP algorithm and / or an MHSA algorithm. By using the query learner to obtain the Q factor, the processing performance of the MHSA layer can be effectively improved, which in turn helps to recover the target message subsequently.

[0091] In some cases, the query learner can adopt an MLP algorithm. For example, the query learner can include at least one MLP layer and does not include an MHSA layer.

[0092] In some cases, the query learner can adopt an MHSA algorithm. For example, the query learner can include at least one MHSA layer and does not include an MLP layer.

[0093] In some cases, the query learner may adopt the MLP algorithm and the MHSA algorithm. For example, the MLP algorithm may first be used to process a set of target image vectors, and then the MHSA algorithm may further process the output of the MLP algorithm to obtain the Q factor. In this case, the query learner may include at least one MLP layer and at least one MHSA layer, and at least one MLP layer may be located before at least one MHSA layer.

[0094] In some implementation manners, if the query learner adopts the MHSA algorithm, the position information of the vectors to be processed by the MHSA algorithm may be input into the MHSA algorithm together.

[0095] In some implementation manners, a set of target image vectors may be processed by multiple MHSA layers to obtain a set of message reconstruction vectors. The multiple MHSA layers may be arranged sequentially. For example, the first MHSA layer among the multiple MHSA layers may process based on a set of target image vectors. Each MHSA layer except the first MHSA layer may process based on the output of the previous MHSA layer of this MHSA layer. The last MHSA layer may output a set of message reconstruction vectors.

[0096] In this case, the respective K factor, V factor, and Q factor may be processed in each MHSA layer to obtain a set of output vectors. It can be understood that the set of output vectors obtained from the last MHSA layer may be used as the above-mentioned set of message reconstruction vectors.

[0097] Specifically, in the first MHSA layer, the K factor, V factor, and Q factor of the first MHSA layer may be determined based on a set of target image vectors. For example, different transformations (such as linear transformations) may be performed on a set of target image vectors to respectively obtain the K factor and V factor of the first MHSA layer.

[0098] For the Q factor, multiple ways may be adopted to obtain it. In some implementation manners, a set of target image vectors may be transformed (such as a linear transformation) to obtain the Q factor of the first MHSA layer. In some implementation manners, the query learner may be used to process a set of target image vectors to obtain the Q factor of the first MHSA layer. The query learner may adopt the MLP algorithm and / or the MHSA algorithm. In this way, the processing performance of the first MHSA layer can be improved.

[0099] The implementation manner of the query learner mentioned here is similar to the implementation manner of the query learner mentioned above, and will not be elaborated here.

[0100] In any MHSA layer except the first MHSA layer, the K factor, V factor, and Q factor of this MHSA layer can be determined based on a set of output vectors of the previous MHSA layer of this MHSA layer.

[0101] For example, different transformations (such as linear transformation) can be performed on a set of output vectors of the previous MHSA layer of this MHSA layer to respectively obtain the K factor, V factor, and Q factor of this MHSA layer.

[0102] For another example, different ways can be used to obtain the Q factor. For example, a query learner can be used to process a set of output vectors of the previous MHSA layer of this MHSA layer to obtain the Q factor of this MHSA layer. The query learner can adopt the MLP algorithm and / or the MHSA algorithm. In this way, the processing performance of this MHSA layer can be improved.

[0103] The implementation manner of the query learner mentioned here is similar to the implementation manner of the query learner mentioned above, and will not be elaborated here.

[0104] It should be understood that different MHSA layers can use the same or different ways to determine the Q factor. For example, for different MHSA layers, a query learner can be used, or a query learner can be not used. In addition, the query learners used for different MHSA layers can be implemented in the same or different ways.

[0105] In addition, as mentioned above, the MHSA algorithm usually considers the positions or orders of the individual vectors in a set of vectors when performing processing. In this case, when using the MHSA layer for processing, the position information of the vectors to be processed by the MHSA layer can also be input into the MHSA layer together.

[0106] After obtaining a set of message reconstruction vectors, a target message can be generated based on the set of message reconstruction vectors. In some embodiments, the MHSA algorithm and / or the MLP algorithm can be used to process the set of message reconstruction vectors to obtain the target message.

[0107] For example, the MLP algorithm can be used to process a set of message reconstruction vectors to obtain the target message. For instance, at least one MLP layer can be used to process the set of message reconstruction vectors to obtain the target message.

[0108] For another example, the MHSA algorithm can be used to process a set of message reconstruction vectors to obtain the target message. For instance, at least one MHSA layer can be used to process the set of message reconstruction vectors to obtain the target message.

[0109] For another example, the MHSA algorithm and the MLP algorithm can be used to process a set of message reconstruction vectors to obtain a target message. In this case, the MHSA algorithm can be first used to process a set of message reconstruction vectors, and then the MLP algorithm can be used to process the output obtained by the MHSA algorithm to obtain the target message. For example, at least one MHSA layer and at least one MLP layer can be used to process a set of message reconstruction vectors to obtain the target message. At least one MHSA layer can be located before at least one MLP layer.

[0110] In addition, if the MHSA algorithm is used when processing a set of message reconstruction vectors, the position information of the vectors to be processed by the MHSA algorithm can also be input into the MHSA algorithm together.

[0111] In some implementation manners, for efficient processing, the encoding manner of the position information involved in the entire process of Figure 5 can be the same as the encoding manner of the position information when encoding the target message in Figure 4 For example, the encoding manner of the position information (such as an encoding function) can be predefined, so that the same encoding manner can be used to encode the position information on both the encoding device side and the decoding device side.

[0112] It can be seen that through the above process, the target message can be efficiently and qualitatively recovered from the target image.

[0113] For easy understanding, the above embodiments will be further described below in conjunction with specific examples. It should be understood that the following examples are only for helping those skilled in the art to better understand the content of this article, rather than limiting the scope of the technical solutions of the present disclosure.

[0114] Figure 6A is a schematic diagram of an example of the functional architecture of an encoding device when implementing image processing according to some embodiments.

[0115] In Figure 6A 's example, in terms of the functional architecture, the encoding device 110 can include an image encoding module 601, a message encoding module 602, a fusion module 603, a reconstruction module 604, and a generation module 605. These modules can be implemented through the structure in Figure 2 .

[0116] As Figure 6AAs shown, the image encoding module 601 can encode the original image 630 to obtain a set of original image vectors 635. The message encoding module 602 can encode the target message 650 to obtain a set of target message vectors 655. The fusion module 603 can process a set of original image vectors 635 and a set of target message vectors 655 to obtain a set of fusion vectors 670. The reconstruction module 604 can reconstruct the residual image 680 based on a set of fusion vectors 670. The generation module 605 can generate the target image 690 containing the target message 650 based on the residual image 680 and the original image 630.

[0117] The operations of each functional module will be further described below in conjunction with specific examples. Figures 6B to 6G Illustrates the use of Figure 6A An exemplary process of performing image processing by each module of.

[0118] As Figure 6B shown, the image encoding module 601 can encode the original image 630. For example, the image encoding module 602 can use a convolutional algorithm or an MLP algorithm to encode the original image 630 to obtain a set of original image vectors 635. For example, the image encoding module 601 can convert the original image 630 into a feature map, and then obtain a set of original image vectors 635 based on the feature map.

[0119] In Figure 6B the example of, a set of original image vectors 635 can include original image vectors 635-1, 635-2, 635-3, etc. It should be understood that the number of original image vectors shown here is only for illustrative purposes, and in different implementation manners, the number of original image vectors may be different, which depends on factors such as specific application scenarios and business requirements.

[0120] As Figure 6C shown, the message encoding module 602 can encode the target message 650 to obtain a set of target message vectors 655. For ease of understanding, in Figure 6C the example of, a set of target message vectors 655 can include target message vectors 655-1, 655-2, 655-3, etc. It should be understood that the number of target message vectors shown here is only for illustrative purposes, and in different implementation manners, the number of target message vectors may be different, which depends on factors such as specific application scenarios and business requirements.

[0121] In addition, in Figure 6CIn the example, the target message 650 may have a binary format and have multiple message segments. Each message segment may be represented by a binary sequence (e.g., a sequence of 0s and / or 1s). The number of bits in each message segment may be greater than or equal to 2. Processing the target message 650 by message segments can improve the processing efficiency of the target message. In Figure 6C the example, the target message 650 may include message segment 01010001, message segment 11000101, message segment 10011010, and so on. The number of bits that the target message 650 has may be D times the total number of pixels of the original image 630, where D is greater than 0. The multiple D may be an integer or may not be an integer. In some cases, for the convenience of expressing the message hiding density, D may be a positive integer greater than 0.

[0122] In some implementations, the total number of message segments may be equal to the total number of pixels of the original image 630 or less than or equal to the total number of pixels of the feature map obtained from the original image 630. The number of bits in each message segment may be greater than or equal to 2. For example, assume the size of the original image 630 is H_im * W_im * D_im, where H_im may represent the length of the original image 630, W_im may represent the width of the original image 630, and D_im may represent the number of channels of the original image 630. Assume the size of the target message 650 can be expressed as H_msg * W_msg * D_msg, where H_msg * W_msg may represent the total number of message segments in the target message 650, and D_msg may represent the number of bits in each message segment. In one case, H_msg * W_msg = H_im * W_im. The case of dividing the message segments with reference to the feature map is similar and will not be elaborated here.

[0123] The message encoding module 602 may encode each message segment of the target message 650 using the MLP algorithm or using the MLP algorithm and the MHSA algorithm to obtain a set of target message vectors 655. For example, the MLP algorithm or the MLP algorithm and the MHSA algorithm may be used to encode each message segment to obtain a corresponding target message vector.

[0124] For ease of understanding, in Figure 6C a further example of the encoding process performed by the message encoding module 602 is shown. As Figure 6CAs shown, the message encoding module 602 may include an MLP layer 602-1 and an MHSA layer 602-2. The MLP layer 602-1 may be located before the MHSA layer 602-2. That is, the MLP layer 602-1 may first process each message segment of the target message 650, and then the MHSA layer 602-2 may further process the output of the MLP layer 602-1. In some implementations, the MHSA layer 602-2 may further process the output of the MLP layer 602-1 based on the position information 651 of each message segment, so as to obtain a set of target message vectors 655. The position information 651 may include a set of position vectors, and each position vector corresponds to a message segment to represent the position or order of the message segment.

[0125] Although not shown in Figure 6C , the message encoding module 602 may be implemented in other ways. For example, the message encoding module 602 may include one or more MLP layers and not include the MHSA layer. For another example, the message encoding module 602 may include one or more MLP layers and one or more MHSA layers, where one or more MLP layers may be located before one or more MHSA layers.

[0126] As Figure 6D shown, the fusion module 603 may include a vector addition sub-module 603-1 and an MHSA sub-module 603-2.

[0127] The vector addition sub-module 603-1 may perform a vector addition operation on a set of original image vectors 635 and a set of target message vectors 655 to obtain a set of input vectors 660. In Figure 6C the example of, a set of input vectors 660 may include an input vector 660-1, an input vector 660-2, an input vector 660-3, and so on.

[0128] As mentioned above, the number of a set of target message vectors 655 may be less than or equal to the number of a set of original image vectors 635. If the number of a set of target message vectors 655 is equal to the number of a set of original image vectors 635, the vector addition sub-module 603-1 may add a set of original image vectors 635 and a set of target message vectors 655 respectively to obtain a set of input vectors 660. For example, as Figure 6D shown, the vector addition sub-module 603-1 may add the original image vector 635-1 and the target message vector 655-1 to obtain the input vector 660-1; the vector addition sub-module 603-1 may add the original image vector 635-2 and the target message vector 655-2 to obtain the input vector 660-2; and so on.

[0129] If the number of a set of target message vectors 655 is less than the number of a set of original image vectors 635, that is to say, a set of original image vectors 635 will be more than a set of target message vectors 655. Then, after performing an addition operation on all the vectors in a set of target message vectors 655, there will still be some original image vectors remaining, and these original image vectors do not have corresponding target message vectors. In this case, a set of input vectors 660 can be regarded as including two parts. The first part of the input vectors can be obtained by adding the original image vectors and the target message vectors, while the second part of the input vectors can be the original image vectors themselves remaining after adding all the target message vectors.

[0130] It should be understood that for the convenience of subsequent processing by the MHSA sub-module 603-2, a set of original image vectors 635 and a set of target message vectors 655 can be added in sequence, so that a set of input vectors 660 also has a corresponding sequence.

[0131] The MHSA sub-module 603-2 can process a set of input vectors 660 using the MHSA algorithm to obtain a set of fused vectors 670. For example, the MHSA sub-module 603-2 can process a set of input vectors 660 in combination with the sequence of a set of input vectors 660 to obtain a set of fused vectors 670. In Figure 6D the example, a set of fused vectors 670 can include fused vectors 670-1, 670-2, 670-3, etc. Generally, the number of a set of original image vectors 635, the number of a set of input vectors 660, and the number of a set of fused vectors 670 can be the same.

[0132] The MHSA sub-module 603-2 can implement the MHSA algorithm in various ways. FIG. 6E(1) shows an implementation example of the MHSA sub-module 603-2, and FIG. 6E(2) shows another implementation example of the MHSA sub-module 603-2.

[0133] In the example of FIG. 6E(1), the MHSA sub-module 603-2 can include an MHSA layer 613.

[0134] In the MHSA layer 613, the K factor and the V factor can be determined based on a set of input vectors 660. For example, different transformations (such as linear transformations) can be performed on a set of input vectors 660 to obtain the K factor and the V factor. In addition, in the MHSA layer 613, the Q factor can be determined based on a set of target message vectors 655. For example, a transformation (such as a linear transformation) can be performed on a set of target message vectors 655 to determine the Q factor.

[0135] Although not shown in FIG. 6E, other ways can also be adopted to determine the Q factor. For example, the Q factor can be determined based on a set of input vectors 660 or a set of original image vectors 635. For instance, a set of input vectors 660 can be transformed to obtain the Q factor. For instance, a set of original image vectors 635 can be transformed to obtain the Q factor.

[0136] Then, in the MHSA layer 613, the K factor, V factor, and Q factor can be processed to obtain a set of fused vectors 670.

[0137] In the example of FIG. 6E(2), the MHSA sub-module 603-2 can include a plurality of MHSA layers arranged in sequence. For example, MHSA layers 613-1, 613-2, ……, 613-M. Except for the first layer 613-1, each subsequent MHSA layer can be processed based on the output of the previous MHSA layer.

[0138] In each MHSA layer, its respective K factor, V factor, and Q factor can be determined, and processed based on its respective K factor, V factor, and Q factor to obtain a set of output vectors. The set of output vectors of the last MHSA layer (for example, in the example of FIG. 6E(2), MHSA layer 613-M) can be used as a set of fused vectors 670.

[0139] In the first MHSA layer 613-1, the K factor, V factor, and Q factor can be determined in a manner similar to the MHSA layer 613 in the example of FIG. 6E(1). For example, the K factor and V factor can be determined based on a set of input vectors 660. For example, a set of input vectors 660 can be subjected to different transformations (such as linear transformation) to obtain the K factor and V factor.

[0140] In addition, in the MHSA layer 613-1, the Q factor can be determined based on a set of target message vectors 655. For example, a set of target message vectors 655 can be transformed (such as linear transformation) to determine the Q factor. Although not shown in FIG. 6E, other ways can also be adopted to determine the Q factor. For example, the Q factor can be determined based on a set of input vectors 660 or a set of original image vectors 635. For instance, a set of input vectors 660 can be transformed to obtain the Q factor. For instance, a set of original image vectors 635 can be transformed to obtain the Q factor.

[0141] In the MHSA layer 613-1, its K factor, V factor, and Q factor can be processed to obtain a set of output vectors 661, for example, output vectors 661-1, 661-2, 661-3, etc.

[0142] For MHSA layers 613-2 to 613-M, the methods for determining the K factor and V factor are similar. For example, for any one of the MHSA layers from MHSA layer 613-2 to MHSA layer 613-M, its K factor and V factor can be determined based on a set of output vectors of the previous MHSA layer of this MHSA layer and a set of target message vectors 655. For example, a vector addition operation can be performed on a set of output vectors of the previous MHSA layer of this MHSA layer and a set of target message vectors to obtain a set of vectors to be processed for this MHSA layer. Then, this set of vectors to be processed can be transformed (such as a linear transformation) to determine its K factor and V factor. For example, in the example of FIG. 6E(2), for MHSA layer 613-2, a vector addition operation can be performed on a set of output vectors 661 of MHSA layer 613-1 and a set of target message vectors 655 to obtain a set of vectors to be processed 622 (such as, vectors to be processed 662-1, 662-2, 662-3, etc.). The K factor and V factor of MHSA layer 613-2 can be determined based on a set of vectors to be processed 622. For example, different transformations can be performed on a set of vectors to be processed 622 to obtain the K factor and V factor of MHSA layer 613-2.

[0143] For any one of the MHSA layers from MHSA layer 613-2 to MHSA layer 613-M, there are multiple ways to determine its Q factor. The ways for these MHSA layers to determine their Q factors can be the same as or different from each other. For example, for this MHSA layer, its Q factor can be determined based on a set of vectors to be processed, a set of target message vectors, or a set of original image vectors of this MHSA layer. For example, a transformation can be performed on a set of vectors to be processed to obtain its Q factor. For another example, a transformation can be performed on a set of target message vectors to obtain its Q factor. For yet another example, a transformation can be performed on a set of original image vectors to obtain its Q factor.

[0144] For example, in the example of FIG. 6E(2), in MHSA layer 613-2, its Q factor can be obtained based on a set of target message vectors 655.

[0145] After the last layer, MHSA layer 613-M processes its K factor, V factor, and Q factor, a set of fused vectors 670 can be obtained.

[0146] After obtaining a set of fused vectors 670, as Figure 6F shown, the reconstruction module 604 can reconstruct the residual image 680 based on a set of fused vectors 670. The reconstruction module 604 can adopt various applicable image reconstruction algorithms, which are not limited herein.

[0147] As Figure 6GAs shown, the generation module 605 may add the residual image 680 to the original image 630 to obtain the target image 690. The target image 690 may contain the target message 650 in a completely invisible or substantially invisible manner.

[0148] The process at the encoding device has been introduced above. An example of image processing at the decoding device will be introduced below.

[0149] Figure 7A FIG. is a schematic diagram of an example of the functional architecture of a decoding device when implementing image processing according to some embodiments.

[0150] In Figure 7A example, in terms of the functional architecture, the decoding device 120 may include an image decoding module 701, a vector processing module 702, and a message reconstruction module 703. These modules may be implemented through the Figure 3 structure.

[0151] The image decoding module 701 may process the target image 690 to obtain a set of target image vectors 730. The vector processing module 702 may process a set of target image vectors 730 to obtain a set of message reconstruction vectors 740. The message reconstruction module 703 may generate the target message 650 based on a set of message reconstruction vectors 740.

[0152] The operations of each functional module will be further described below with specific examples. Figures 7B to 7D FIG. shows an exemplary process of performing image processing using each Figure 7A module.

[0153] As Figure 7B shown, the image decoding module 701 may process the target image 690 using a convolutional algorithm or an MLP algorithm to obtain a set of target image vectors 730. For example, the image decoding module 701 may convert the target image 690 into a feature map, and then obtain a set of target image vectors 730 based on the feature map.

[0154] In Figure 7B example, a set of target image vectors 730 may include original image vectors 730-1, 730-2, 730-3, etc. It should be understood that the number of target image vectors shown here is only for illustrative purposes, and in different implementations, the number of target image vectors may be different, depending on factors such as specific application scenarios and business requirements.

[0155] As shown in FIG. 7C, the vector processing module 702 may process a set of target image vectors 730 using the MHSA algorithm to obtain a set of message reconstruction vectors 740. In the example of FIG. 7C, a set of message reconstruction vectors 740 may include message reconstruction vectors 740-1, 740-2, 740-3, and so on.

[0156] The vector processing module 702 may implement the MHSA algorithm in various ways. FIG. 7C(1) shows an implementation example of the vector processing module 702, and FIG. 7C(2) shows another implementation example of the vector processing module 702.

[0157] In the example of FIG. 7C(1), the vector processing module 702 may include an MHSA layer 722.

[0158] In the MHSA layer 722, the K factor and the V factor may be determined based on a set of target image vectors 730. For example, different transformations (such as linear transformations) may be performed on a set of target image vectors 730 to obtain the K factor and the V factor.

[0159] For the MHSA layer 722, its Q factor may be determined based on a set of target image vectors 730. This may be achieved in various ways. In some implementations, different transformations (such as linear transformations) may be performed on a set of target image vectors 730 to obtain its Q factor. In some implementations, the vector processing module 702 may further include a query learner 712. The query learner 712 may process a set of target image vectors 730 to obtain the Q factor. The query learner 712 may adopt the MLP algorithm and / or the MHSA algorithm.

[0160] In the example of FIG. 7C(1), the query learner 712 may include an MLP layer 712-1 and an MHSA layer 712-2. The MLP layer 712-1 may be located before the MHSA layer 712-2. That is to say, the MLP layer 712-1 may first process a set of target image vectors 730, and then the MHSA layer 712-2 may further process the output of the MLP layer 712-1 to obtain the Q factor.

[0161] In addition, although not shown in FIG. 7C(1), it can be understood that the query learner 712 may be implemented in other ways. For example, the query learner 712 may include at least one MLP layer and not include the MHSA layer. For another example, the query learner 712 may include at least one MHSA layer and not include the MLP layer. For another example, the query learner 712 may include at least one MLP layer and at least one MHSA layer, where at least one MLP layer may be located before at least one MHSA layer.

[0162] In the example of FIG. 7C(2), the vector processing module 702 may include a plurality of MHSA layers arranged in sequence, for example, MHSA layers 722-1, 722-2, ……, 722-P. Except for the first layer 722-1, each subsequent MHSA layer may be processed based on the output of the previous MHSA layer.

[0163] In each MHSA layer, respective K factor, V factor, and Q factor may be determined and processed based on the respective K factor, V factor, and Q factor to obtain a set of output vectors. A set of output vectors of the last MHSA layer (e.g., in the example of FIG. 7C(2), MHSA layer 722-P) will be used as a set of message reconstruction vectors 740.

[0164] In the first MHSA layer 722-1, the K factor, V factor, and Q factor may be determined in a manner similar to the MHSA layer 722 in the example of FIG. 7C(1). For example, based on a set of target image vectors 730, the K factor and V factor may be determined. For example, different transformations (such as linear transformations) may be performed on a set of target image vectors 730 to obtain the K factor and V factor. The Q factor of a set of target image vectors 730 may be determined. This can be achieved in various ways. In some implementations, in the MHSA layer 722-1, for example, a transformation (such as a linear transformation) may be performed on a set of target image vectors 730 to obtain its Q factor. In some implementations, the vector processing module 702 may further include a query learner 712-1. The query learner 712-1 may process a set of target image vectors 730 to obtain the Q factor of the MHSA layer 722-1. The query learner 712-1 may employ an MLP algorithm and / or an MHSA algorithm.

[0165] In the MHSA layer 722-1, its K factor, V factor, and Q factor may be processed to obtain a set of output vectors 731, for example, output vectors 731-1, 731-2, 731-3, and so on.

[0166] For the MHSA layers 722-2 to 722-P, the ways to determine the K factor and the V factor are similar. For example, for any one of the MHSA layers from the MHSA layer 722-2 to the MHSA layer 722-P, its K factor and V factor can be determined based on a set of output vectors of the previous MHSA layer of this MHSA layer. For example, a transformation (such as a linear transformation) can be performed on a set of output vectors of the previous MHSA layer of this MHSA layer to determine its K factor and V factor. For example, in the example of FIG. 7C(2), in the MHSA layer 722-2, its K factor and V factor can be determined based on a set of output vectors 731 of the MHSA layer 722-1. For example, a transformation can be performed on the set of output vectors 731 of the MHSA layer 722-1 to obtain its K factor and V factor.

[0167] For any one of the MHSA layers from the MHSA layer 722-2 to the MHSA layer 722-M, there are multiple ways to determine its Q factor. The ways for these MHSA layers to determine their Q factors can be the same as or different from each other. For example, for this MHSA layer, a transformation can be performed on a set of output vectors of the previous MHSA layer of this MHSA layer to obtain its Q factor. For another example, a query learner can be used to process a set of output vectors to obtain its Q factor. Similarly, the query learner can adopt the MLP algorithm and / or the MHSA algorithm.

[0168] For example, in the example of FIG. 7C(2), for the MHSA layer 722-2, a set of output vectors 731 can be processed by the query learner 712-2 to obtain its Q factor.

[0169] In the example of FIG. 7C(2), the query learner 712-1 can include an MLP layer 712-1(1) and an MHSA layer 712-1(2). The MLP layer 712-1(1) can be located before the MHSA layer 712-1(2). That is to say, the MLP layer 712-1(1) can first process a set of target image vectors 730, and then the MHSA layer 712-1(2) can further process the output of the MLP layer 712-1(1) to obtain the Q factor.

[0170] Similarly, the query learner 712-2 can include an MLP layer 712-2(1) and an MHSA layer 712-2(2).

[0171] In addition, although not shown in FIG. 7C(2), it can be understood that the query learner 712-1 or 712-2 can be implemented in other ways. For example, the query learner 712-1 or 712-2 can include at least one MLP layer and not include an MHSA layer. For another example, the query learner 712-1 or 712-2 can include at least one MHSA layer and not include an MLP layer. For yet another example, the query learner 712-1 or 712-2 can include at least one MLP layer and at least one MHSA layer, where at least one MLP layer can be located before at least one MHSA layer.

[0172] After the last layer, the MHSA layer 722-P processes its K factor, V factor, and Q factor, a set of message reconstruction vectors 740 can be obtained.

[0173] As Figure 7D described above, the message reconstruction module 730 can generate the target message 650 based on a set of message reconstruction vectors 740. The message reconstruction module 730 can use the MHSA algorithm and / or the MLP algorithm to generate the target message 650 based on a set of message reconstruction vectors 740. The obtained target message 650 can have a binary format and have multiple message segments.

[0174] In Figure 7D the example of, the message reconstruction module 703 can include an MHSA layer 703-1 and an MLP layer 703-2. The MHSA layer 703-1 can be located before the MLP layer 703-2. That is to say, the MHSA layer 703-1 can first process a set of message reconstruction vectors 740, and then the MLP layer 703-2 can further process the output of the MHSA layer 703-1 to obtain the target message 650.

[0175] Although not shown in Figure 7D it, the message reconstruction module 703 can be implemented in other ways. For example, the message reconstruction module 703 can include at least one MHSA layer and not include an MLP layer. For another example, the message reconstruction module 703 can include at least one MLP layer and not include an MHSA layer. For yet another example, the message reconstruction module 703 can include at least one MHSA layer and at least one MLP layer, where at least one MHSA layer can be located before at least one MLP layer.

[0176] Figure 8 An example showing the comparison between the original image and the target image is shown. In Figure 8In the example, after hiding the message in the original image 801-1, the target image 801-2 is obtained, and the message hiding density is 3 bits per pixel. Similarly, after hiding the message in the original image 802-1, the target image 802-2 is obtained, and the message hiding density is 3 bits per pixel. It can be seen that the embodiments of the present disclosure can achieve good message hiding performance.

[0177] Figure 9 is a schematic block diagram of a device for image processing according to some embodiments. The device 900 may correspond to Figure 1 the encoding device 110 in

[0178] As Figure 9 shown, the device 900 may include an image vector unit 902, a message vector unit 904, a first image generation unit 906, and a second image generation unit 908.

[0179] The image vector unit 902 may generate a set of original image vectors based on the original image. The message vector unit 904 may generate a set of target message vectors based on the target message. The first image generation unit 906 may process a set of original image vectors and a set of target message vectors based on the MHSA algorithm to generate a residual image. The second image generation unit 908 may generate a target image based on the residual image and the original image, where the target image may contain the target message.

[0180] Each unit of the device 900 may execute the specific processes described above for the encoding device. Therefore, for the sake of brevity of description, the specific operations and functions of each unit of the device 900 are not described herein again.

[0181] Figure 10 is a schematic block diagram of a device for image processing according to some embodiments. The device 900 may correspond to Figure 1 the decoding device 120 in

[0182] As Figure 10 shown, the device 1000 may include an image processing unit 1002, a vector generation unit 1004, and a message generation unit 1006.

[0183] The image processing unit 1002 may process the target image to generate a set of target image vectors, where the target image contains the target message. The vector generation unit 1004 may process a set of target image vectors based on the MHSA algorithm to generate a set of message reconstruction vectors. The message generation unit 1006 may generate the target message based on a set of message reconstruction vectors.

[0184] Each unit of the apparatus 1000 may perform the specific processes described above with respect to the decoding device. Therefore, for the sake of brevity of description, the specific operations and functions of each unit of the apparatus 1000 will not be described herein again.

[0185] Embodiments of the present disclosure also provide a computer-readable storage medium. The computer-readable storage medium may store executable code, and when the executable code is executed, it implements the specific processes described above with respect to the encoding device or the decoding device.

[0186] For example, the computer-readable storage medium may include, but is not limited to, RAM, ROM, Electrically-Erasable Programmable Read-Only Memory (EEPROM), Static Random Access Memory (SRAM), hard disk, flash memory, and the like.

[0187] Embodiments of the present disclosure also provide a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the specific processes described above with respect to the encoding device or the decoding device.

[0188] Specific embodiments of the present disclosure have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the figures do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0189] Not all steps and units in the above processes and system structure diagrams are necessary, and some steps or units may be omitted according to actual needs. The device structures described in the above embodiments may be physical structures or logical structures. That is, some units may be implemented by the same physical entity, some units may be implemented separately by multiple physical entities, or may be jointly implemented by some components in multiple independent devices.

[0190] The optional embodiments of the embodiments of the present disclosure have been described in detail above in conjunction with the accompanying drawings. However, the embodiments of the present disclosure are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present disclosure, various modifications can be made to the technical solutions of the embodiments of the present disclosure, and these modifications all fall within the protection scope of the embodiments of the present disclosure.

Claims

1. A method for image processing, comprising: Generating a set of original image vectors based on an original image; Generating a set of target message vectors based on a target message; Processing the set of original image vectors and the set of target message vectors based on the multi-head self-attention (MHSA) algorithm to generate a residual image; Generating a target image based on the residual image and the original image, wherein the target image contains the target message.

2. The method according to claim 1, wherein, Generating a set of target message vectors, including: Encoding the target message using a multi-layer perceptron (MLP) algorithm to obtain the set of target message vectors; or Encoding the target message using an MLP algorithm and an MHSA algorithm to obtain the set of target message vectors.

3. The method according to claim 2, wherein, Encoding using an MLP algorithm to obtain the set of target message vectors, including: Encoding the target message using at least one MLP layer to obtain the set of target message vectors.

4. The method according to claim 2, wherein Encoding the target message using an MLP algorithm and an MHSA algorithm to obtain the set of target message vectors, including: Encoding the target message using at least one MLP layer and at least one MHSA layer to obtain the set of target message vectors, wherein the at least one MLP layer is located before the at least one MHSA layer.

5. The method according to claim 1, wherein Generating a set of original image vectors, including: Encoding the original image using a convolutional algorithm or an MLP algorithm to obtain the set of original image vectors.

6. The method according to claim 1, wherein, Processing the set of original image vectors and the set of target message vectors using an MHSA algorithm to generate a residual image, including: Performing a vector addition operation on the set of original image vectors and the set of target message vectors to obtain a set of input vectors; Processing the set of input vectors using an MHSA algorithm to obtain a set of fused vectors; Generating the residual image based on the set of fused vectors.

7. The method according to claim 6, wherein The number of the set of target message vectors is equal to the number of the set of original image vectors; Performing a vector addition operation on the set of original image vectors and the set of target message vectors to obtain a set of input vectors, including: Adding the set of original image vectors and the set of target message vectors respectively in a corresponding manner to obtain the set of input vectors.

8. The method according to claim 6, wherein The number of the set of target message vectors is less than the number of the set of original image vectors; Performing a vector addition operation on the set of original image vectors and the set of target message vectors to obtain a set of input vectors, including: Adding a part of the original image vectors in the set of original image vectors to the set of target message vectors to obtain a first part of input vectors, wherein the number of the part of the original image vectors is equal to the number of the set of target message vectors, the set of input vectors includes the first part of input vectors and a second part of input vectors, and the remaining original image vectors in the set of original image vectors except the part of the original image vectors serve as the second part of input vectors.

9. The method according to claim 6, wherein Processing the set of input vectors using an MHSA algorithm to obtain a set of fused vectors, including: Process the set of input vectors using a single MHSA layer to obtain the set of fused vectors.

10. The method according to claim 9, wherein Processing the set of input vectors using a single MHSA layer to obtain the set of fused vectors includes: Perform the following operations in the single MHSA layer: Based on the set of input vectors, determine the key factor and the value factor; Based on the set of input vectors, the set of original image vectors, or the set of target message vectors, determine the query factor; Process the key factor, the value factor, and the query factor to obtain the set of fused vectors.

11. The method according to claim 6, wherein, Process the set of input vectors using the MHSA algorithm to obtain a set of fused vectors, including: Process the set of input vectors using multiple MHSA layers to obtain the set of fused vectors, where: The first MHSA layer in the multiple MHSA layers processes based on the set of input vectors, Each MHSA layer except the first MHSA layer processes based on the output of the previous MHSA layer of this MHSA layer, The last MHSA layer outputs the set of fused vectors.

12. The method according to claim 11, wherein, Processing the set of input vectors using multiple MHSA layers to obtain the set of fused vectors includes: In each MHSA layer of the multiple MHSA layers, process their respective key factors, value factors, and query factors to obtain a set of output vectors, where the set of output vectors obtained from the last MHSA layer is used as the set of fused vectors, where: In the first MHSA layer: Based on the set of input vectors, determine the key factor and the value factor of the first MHSA layer; Based on the set of input vectors, the set of original image vectors, or the set of target message vectors, determine the query factor of the first MHSA layer; In any MHSA layer except the first MHSA layer: Based on a set of output vectors of the previous MHSA layer of this MHSA layer and the set of target message vectors, determine the key factor and the value factor of this MHSA layer; Based on both a set of output vectors of the previous MHSA layer of this MHSA layer and the set of target message vectors, the set of target message vectors, or the set of original image vectors, determine the query factor of this MHSA layer.

13. The method according to claim 12, wherein, In any MHSA layer except the first MHSA layer, determining the key factor, value factor, and query factor of this MHSA layer includes: Perform a vector addition operation on a set of output vectors of the previous MHSA layer of this MHSA layer and the set of target message vectors to obtain a set of vectors to be processed for this MHSA layer; Based on the set of vectors to be processed for this MHSA layer, determine the key factor and the value factor of this MHSA layer; Based on the set of vectors to be processed for this MHSA layer, the set of target message vectors, or the set of original image vectors, determine the query factor of this MHSA layer.

14. The method according to claim 1, wherein, The target message has a binary format, and the total number of bits of the target message is D times the total number of pixels of the original image, where D > 0.

15. The method according to claim 14, wherein, The target message has multiple message segments, each message segment having the same number of bits and the number of bits in each message segment being greater than or equal to; Generate a set of target message vectors, including: Encode the multiple message segments to obtain the set of target message vectors, where each message segment corresponds to a target message vector.

16. The method according to claim 1, wherein Each target message vector in the set of target message vectors and each original image vector in the set of original image vectors have the same length.

17. The method according to claim 1, wherein, Generate a target image, including: Add the residual image to the original image to obtain the target image.

18. The method according to claim 1, wherein The target image presents the same visual content as the original image.

19. The method according to claim 1, wherein, The target image contains the target message in a visually completely invisible manner, or the target image contains the target message with a preset visual visibility level.

20. A method for image processing, including: Process a target image to generate a set of target image vectors, where the target image contains a target message; Process the set of target image vectors based on the multi-head self-attention (MHSA) algorithm to generate a set of message reconstruction vectors; Generate a target message based on the set of message reconstruction vectors.

21. The method according to claim 20, wherein, Generate a set of target image vectors, including: Encode the target image using a convolutional algorithm or an MLP algorithm to obtain the set of target image vectors.

22. The method according to claim 20, wherein Generate a set of message reconstruction vectors, including: Process the set of target image vectors using a single MHSA layer to obtain the set of message reconstruction vectors.

23. The method according to claim 22, wherein, Process the set of target image vectors using a single MHSA layer to obtain the set of message reconstruction vectors, including: Perform the following operations in the single MHSA layer: Based on the set of target image vectors, determine a key factor, a value factor, and a query factor; Process the key factor, the value factor, and the query factor to obtain the set of message reconstruction vectors.

24. The method according to claim 23, wherein, Determine the query factor, including: Process the set of target image vectors using a query learner to obtain the query factor, where the query learner uses an MLP algorithm and / or an MHSA algorithm.

25. The method according to claim 20, wherein Generate a set of message reconstruction vectors, including: Process the set of target image vectors using multiple MHSA layers to obtain the set of message reconstruction vectors, where: The first MHSA layer in the multiple MHSA layers processes based on the set of target image vectors, Each MHSA layer except the first MHSA layer processes based on the output of the previous MHSA layer of this MHSA layer, The last MHSA layer outputs the set of message reconstruction vectors.

26. The method according to claim 25, wherein, Process the set of target image vectors using multiple MHSA layers to obtain the set of message reconstruction vectors, including: Process the respective key factor, value factor, and query factor in each MHSA layer in the multiple MHSA layers to obtain a set of output vectors, where the set of output vectors obtained from the last MHSA layer is used as the set of message reconstruction vectors, where: In the first MHSA layer: Determine the key factor, value factor, and query factor of the first MHSA layer based on the set of target image vectors; In any MHSA layer other than the first MHSA layer: Determine the key factor, value factor, and query factor of this MHSA layer based on a set of output vectors of the previous MHSA layer of this MHSA layer.

27. The method according to claim 26, wherein, Determining the query factor of the first MHSA layer in the first MHSA layer includes: Processing the set of target image vectors by a query learner to obtain the query factor of the first MHSA layer, where the query learner uses an MLP algorithm and / or an MHSA algorithm.

28. The method according to claim 26, wherein Determining the query factor of this MHSA layer in any MHSA layer other than the first MHSA layer includes: Processing a set of output vectors of the previous MHSA layer of this MHSA layer by a query learner to obtain the query factor of this MHSA layer, where the query learner uses an MLP algorithm and / or an MHSA algorithm.

29. The method according to any one of claims 24, 27, or 28, wherein: The query learner includes at least one MLP layer; or The query learner includes at least one MHSA layer; or The query learner includes at least one MLP layer and at least one MHSA layer, where the at least one MLP layer is located before the at least one MHSA layer.

30. The method according to claim 20, wherein, Generating a target message includes: Processing the set of message reconstruction vectors using an MHSA algorithm and / or an MLP algorithm to obtain the target message.

31. The method according to claim 30, wherein, Processing the set of message reconstruction vectors using an MLP algorithm and / or an MHSA algorithm to obtain the target message includes: Processing the set of message reconstruction vectors using at least one MLP layer to obtain the target message; or Processing the set of message reconstruction vectors using at least one MHSA layer to obtain the target message; or Processing the set of message reconstruction vectors using at least one MHSA layer and at least one MLP layer to obtain the target message, where the at least one MHSA layer is located before the at least one MLP layer.

32. The method according to claim 20, wherein, The target image contains the target message in a visually completely invisible manner, or the target image contains the target message with a preset visual visibility level.

33. An apparatus for image processing, comprising: An image vector unit configured to generate a set of original image vectors based on an original image; A message vector unit configured to generate a set of target message vectors based on a target message; A first image generation unit configured to process the set of original image vectors and the set of target message vectors based on an MHSA algorithm to generate a residual image; A second image generation unit configured to generate a target image based on the residual image and the original image, where the target image contains the target message.

34. An apparatus for image processing, comprising: An image processing unit, configured to: process a target image to generate a set of target image vectors, where the target image contains a target message; A vector generation unit, configured to: process the set of target image vectors based on the MHSA algorithm to generate a set of message reconstruction vectors; A message generation unit, configured to: generate a target message based on the set of message reconstruction vectors.

35. An apparatus for image processing, comprising: At least one processor; A memory in communication with the at least one processor, having executable code stored thereon, the executable code, when executed by the at least one processor, causes the at least one processor to execute the method according to any one of claims 1 to 19.

36. An apparatus for image processing, comprising: At least one processor; A memory in communication with the at least one processor, having executable code stored thereon, the executable code, when executed by the at least one processor, causes the at least one processor to execute the method according to any one of claims 20 to 32.

37. A computer-readable storage medium, storing executable code, the executable code, when executed, implements the method according to any one of claims 1 to 32.

38. A computer program product, comprising a computer program, the computer program, when executed by a processor, implements the method according to any one of claims 1 to 32.

Images