Image processing method and device, electronic equipment and medium

Abstract

The invention discloses an image processing method and device, electronic equipment and a medium, and belongs to the technical field of image processing. The method comprises the following steps: receiving an encrypted image corresponding to an original image and encrypted information, wherein the encrypted information is a transformation relation adopted for transforming the original image into the encrypted image; and inputting the encrypted image into an image processing model to perform image feature data reduction operation on the encrypted image based on the encrypted information to obtain image feature data corresponding to the encrypted image, the image feature data corresponding to the encrypted image being the same as the image feature data corresponding to the original image.

Description

Technical Field

[0001] This application belongs to the field of image processing technology, specifically relating to an image processing method, apparatus, electronic device, and medium. Background Technology

[0002] With the development of artificial intelligence technology, especially the widespread deployment of large models, such as visual encoders, in various cloud services, more and more mobile devices (such as smartphones and smart glasses) are beginning to rely on cloud-based artificial intelligence (AI) models for the analysis and processing of multimodal data such as images and videos. These services typically require uploading user data to the cloud for pre-trained models to perform operations such as classification, retrieval, recognition, or summarization.

[0003] However, this "end-to-end data collection - cloud processing" workflow places high demands on user privacy protection. Especially when the uploaded data contains highly sensitive information such as faces, living environments, identity documents, and screenshots, malicious access or breaches of this data in the cloud could lead to serious information leaks and security risks. Summary of the Invention

[0004] The purpose of this application is to provide an image processing method, apparatus, electronic device, and medium that improves privacy and security by restoring image feature data for image processing.

[0005] In a first aspect, embodiments of this application provide an image processing method, including: Receive the encrypted image corresponding to the original image and the encryption information, wherein the encryption information is the transformation relationship used to transform the original image into the encrypted image; The encrypted image is input into an image processing model to perform an image feature data restoration operation on the encrypted image based on the encryption information, thereby obtaining the image feature data corresponding to the encrypted image. The image feature data corresponding to the encrypted image is the same as the image feature data corresponding to the original image.

[0006] Secondly, embodiments of this application provide an image processing method, including: Acquire the original image and the encrypted information that encrypts the original image; The original image is transformed into an encrypted image according to the transformation relationship in the encrypted information; Send the encrypted image corresponding to the original image, along with the encrypted information.

[0007] Thirdly, embodiments of this application provide an image processing apparatus, including: A receiving module is used to receive an encrypted image corresponding to the original image and encrypted information, wherein the encrypted information is the transformation relationship used to transform the original image into the encrypted image; The restoration module is used to input the encrypted image into the image processing model to perform an image feature data restoration operation on the encrypted image based on the encryption information, so as to obtain the image feature data corresponding to the encrypted image. The image feature data corresponding to the encrypted image is the same as the image feature data corresponding to the original image.

[0008] Fourthly, embodiments of this application provide an image processing apparatus, including: The acquisition module is used to acquire the original image and the encrypted information that encrypts the original image; A transformation module is used to transform the original image into an encrypted image according to the transformation relationship in the encrypted information; The sending module is used to send the encrypted image corresponding to the original image and the encrypted information to the network device.

[0009] Fifthly, embodiments of this application provide an electronic device including a processor and a memory, the memory storing programs or instructions executable on the processor, the programs or instructions, when executed by the processor, implementing the steps of the method described in the first or second aspect.

[0010] In a sixth aspect, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first or second aspect.

[0011] In a seventh aspect, embodiments of this application provide a chip, the chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the methods described in the first or second aspect.

[0012] Eighthly, embodiments of this application provide a computer program product stored in a storage medium, which is executed by at least one processor to implement the method as described in the first or second aspect.

[0013] In this embodiment, if an encrypted image corresponding to the original image and the encryption information used to transform the original image into the encrypted image are received, the encrypted image is input into the image processing model. Based on the encryption information, an image feature data restoration operation is performed on the encrypted image to obtain image feature data corresponding to the encrypted image that is identical to the image feature data corresponding to the original image. The above scheme does not require restoring the original image, but only restores the image feature data, thus improving privacy and security. Furthermore, by performing the image feature data restoration operation on the encrypted image based on the encryption information, only the parameters used in the image feature extraction process in the image processing model need to be adjusted. There is no need to modify the model weights or structure, resulting in lower computational costs and good versatility and adaptability. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating the edge-cloud image processing method of existing approaches; Figure 2 This is a schematic flowchart of an image processing method provided in some embodiments of this application; Figure 3 This is a schematic flowchart of edge-cloud image processing provided in some embodiments of this application; Figure 4 These are schematic diagrams illustrating patch-level image encryption provided in some embodiments of this application; Figure 5 This is a schematic diagram of patch-level image encryption represented in the form of a two-dimensional grid, provided in some embodiments of this application; Figure 6 These are schematic diagrams illustrating pixel-level image encryption provided in some embodiments of this application; Figure 7 These are schematic diagrams illustrating pixel-level image encryption in the form of a two-dimensional grid, provided in some embodiments of this application. Figure 8 These are schematic diagrams illustrating Patch-level and Pixel-level image encryption provided in some embodiments of this application; Figure 9 This is a schematic diagram of image encryption at the Patch and Pixel levels, provided in the form of a two-dimensional grid according to some embodiments of this application; Figure 10 These are schematic diagrams of images before and after pixel-level encryption provided in some embodiments of this application; Figure 11 These are schematic diagrams illustrating the specific process of pixel-level image processing provided in some embodiments of this application; Figure 12 These are schematic diagrams of images before and after Patch-level encryption provided in some embodiments of this application; Figure 13This is a schematic diagram of a specific process for Patch-level image processing provided in some embodiments of this application; Figure 14 These are schematic diagrams of images before and after Pixel-level and Patch-level encryption provided in some embodiments of this application; Figure 15 These are schematic diagrams illustrating specific processes for pixel-level and patch-level image processing provided in some embodiments of this application; Figure 16 This is a schematic flowchart of an image processing method provided in some embodiments of this application; Figure 17 These are schematic diagrams of the structure of an image processing apparatus provided in some embodiments of this application; Figure 18 These are schematic diagrams of the structure of an image processing apparatus provided in some embodiments of this application; Figure 19 These are structural block diagrams of electronic devices provided in some embodiments of this application; Figure 20 These are structural block diagrams of electronic devices provided in some embodiments of this application. Detailed Implementation

[0015] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0016] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0017] Currently, traditional symmetric encryption (such as Advanced Encryption Standard (AES)), homomorphic encryption, differential privacy, federated learning, and model-layer fusion encryption, among other privacy protection methods, have improved the security of data transmission and processing to some extent. However, they still have many limitations and cannot meet the dual requirements of privacy protection and model compatibility in practical applications. Although various privacy protection mechanisms exist for image / video data processing between mobile devices and cloud services, these solutions still have significant shortcomings and limitations in practical applications, specifically in the following aspects: 1. Traditional encryption and decryption processes pose security risks due to data exposure in the cloud. Currently, widely used symmetric or asymmetric encryption algorithms mostly rely on a process of "end-side encryption + cloud-side decryption + cloud-based processing". Although such methods can protect data from being intercepted during transmission, once the data is uploaded to the cloud server and decrypted for use in model inference, the original content is exposed in the cloud memory or intermediate cache. If the server suffers a hacker attack, privilege leak, or system vulnerability, it may cause a large-scale privacy leak.

[0018] Furthermore, traditional encryption is "transparent" to model processing, meaning encryption and AI inference are completely decoupled. This necessitates plaintext decryption in the cloud, creating inherent vulnerabilities, such as... Figure 1 As shown, the terminal device encrypts the original image, and the encrypted image is transmitted to the network device. The network device decrypts and restores the original image, and then inputs the original image into the image processing model for processing.

[0019] 2. The model-integrated encryption method is complex to implement and has poor versatility. Methods that embed encryption operations into the model itself, such as using reversible neural networks, obfuscation layers, and visual mixers, can reduce plaintext exposure, but they typically have the following drawbacks: High structural coupling: Most fusion solutions require customized modifications to the model architecture, such as rewriting the input layer, feature fusion module, or subsequent attention mechanism. They are not suitable for mainstream pre-trained large models, such as: Vision Transformer (ViT) and Contrastive Language–Image Pre-training (CLIP).

[0020] Model retraining is required: After encryption is integrated, the accuracy of the model decreases, and it is often necessary to retrain or even jointly train the visual model and the encryption module, which consumes huge training resources.

[0021] Poor versatility: The encryption encoding is strongly coupled with the model, and migrating to other models requires redesigning the mapping scheme, resulting in high engineering costs, such as migrating from ViT to modern convolutional networks (Convolutional Next, ConvNeXt) or segmentation models (Segment Anything Model, SAM).

[0022] 3. It is difficult to strike a balance between encryption strength and model usability. High-strength encryption is often achieved by significantly scrambling image pixels (e.g., pixel scrambling, masking, blurring, etc.), but this will seriously damage the structural and semantic nature of image features, making it impossible for visual models to effectively extract key features, resulting in a significant decrease in inference accuracy; conversely, if the encryption perturbation is small, it may not be able to effectively prevent the human eye or attackers from reconstructing the original image, resulting in poor privacy protection.

[0023] Therefore, the embodiments of this application provide an image processing method, apparatus, electronic device and medium that does not require restoring the original image, but only restores the image feature data. Moreover, the restoration process only requires adjusting the parameters used in the image feature extraction process in the image processing model, which not only improves privacy and security, but also does not require modification of model weights or structure, has low computational cost, and has good versatility and adaptability.

[0024] The image processing method provided in this application embodiment can be applied to image processing scenarios of network devices, that is, to perform image processing on encrypted images sent by terminal devices; it can also be applied to image processing scenarios of terminal devices, that is, to perform image processing on encrypted images sent by network devices.

[0025] The image processing method provided in this application will be described below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0026] like Figure 2 As shown in the figure, this application provides an image processing method, which may specifically include the following steps: Step 201: Receive the encrypted image corresponding to the original image and the encryption information, wherein the encryption information is the transformation relationship used to transform the original image into the encrypted image.

[0027] Specifically, when the terminal device performs encryption, it transforms the original image into an encrypted image according to the transformation relationship in the encryption information. This encryption process effectively encrypts the original image while maintaining the overall image structure. The terminal device packages the encryption information and the encrypted image together and uploads them to the network device. The network device receives the encrypted image of the original image and the encryption information sent by the terminal device.

[0028] The original image can also be extended to multimodal data such as video, depth map, and semantic map.

[0029] Step 202: Input the encrypted image into the image processing model to perform an image feature data restoration operation on the encrypted image based on the encryption information, so as to obtain the image feature data corresponding to the encrypted image. The image feature data corresponding to the encrypted image is the same as the image feature data corresponding to the original image.

[0030] Specifically, the received encrypted image is input into the image processing model. The image processing model performs an image feature data restoration operation on the encrypted image based on the encryption information, so that the image feature data extracted from the encrypted image is the same as the image feature data extracted from the original image input into the image processing model, thereby achieving the purpose of restoring the image feature data.

[0031] The above scheme, such as Figure 3 As shown, the original image is encrypted on the terminal device to obtain an encrypted image, which is then transmitted to the network device. The network device directly inputs the encrypted image into the image processing model for processing. During the entire processing, the original image is not restored; only the image feature data corresponding to the original image is restored. Then, the image feature data is processed to obtain the processing result and output it. This ensures that the entire processing process is unreadable and the original image cannot be restored, thereby effectively avoiding the privacy risk of the network device exposing the original image in the traditional "encryption-decryption-recognition" process.

[0032] In another embodiment, after the network device receives the processing result, it can determine whether encryption is required before transmitting the data to the terminal device based on the specific processing result. For example, if the processing result is the generation of text for video understanding and description, the network device can transmit the text to the terminal device without encryption. If the processing result is an edited video, the network device can set whether to encrypt or not the edited video as needed. If encryption is required, the edited video is encrypted according to the terminal device's encryption method (i.e., transforming each frame of the edited image to an encrypted image according to the transformation relationship in the encrypted information) before being transmitted to the terminal device. The terminal device then decrypts the video according to the network device's decryption method (i.e., the method in step 202) to improve privacy and security.

[0033] In this embodiment, if an encrypted image corresponding to the original image and the encryption information used to transform the original image into the encrypted image are received, the encrypted image is input into the image processing model. Based on the encryption information, an image feature data restoration operation is performed on the encrypted image to obtain image feature data corresponding to the encrypted image that is identical to the image feature data corresponding to the original image. The above scheme does not require restoring the original image, but only restores the image feature data, thus improving privacy and security. Furthermore, by performing the image feature data restoration operation on the encrypted image based on the encryption information, only the parameters used in the image feature extraction process in the image processing model need to be adjusted. There is no need to modify the model weights or structure, resulting in lower computational costs and good versatility and adaptability.

[0034] In an optional specific embodiment, when the encrypted information includes the image patch position transformation relationship of the original image, the encrypted image is an image obtained by transforming the image patches of the original image based on the image patch position transformation relationship; and / or, When the encrypted information includes the pixel position transformation relationship in each image block of the original image, the encrypted image is an image obtained by transforming the pixel positions in each image block of the original image based on the pixel position transformation relationship.

[0035] In one embodiment, if the encrypted information includes the image block position transformation relationship of the original image, the terminal device transforms the image blocks in the original image based on the image block position transformation relationship during encryption processing to obtain the encrypted image. Thus, the network device can determine the correspondence between the positions of the image blocks in the original image and the positions of the image blocks in the encrypted image through the encrypted information. Since the transformation from the original image to the encrypted image is merely a transformation of image block positions, this encryption method is simple and can reduce computational overhead, latency, and power consumption.

[0036] For example, the encrypted information includes a two-dimensional arrangement of image patch positions in the original image, which is used to encode the positional order of the individual image patches in the original image. In a typical ViT encoder, the original image is usually divided into a 14x14 patch grid; therefore, the image patch position transformation can be represented as a 14x14 position mapping to control the arrangement order of patches in the input sequence. Figure 4As shown, taking the original image as a 4*4 patch grid as an example, when the terminal device encrypts the image, it performs a corresponding temporal transformation operation on the original image according to the image patch position transformation relationship, thereby scrambling the original spatial order of each patch in the original image to obtain an encrypted image. This process is essentially rearranging the structure of the original image at the patch level to achieve effective encryption of the original image content while maintaining the compatibility of the visual encoder.

[0037] like Figure 5 As shown, the above image block position transformation can be represented by a two-dimensional grid. If the original image is divided into image blocks 1 to 16, image block 1 is located at image block position 1, image block 2 is located at image block position 2, ..., and image block 16 is located at image block position 16. The image block position transformation relationship means transforming image block 1 to image block position 15, transforming image block 2 to image block position 7, transforming image block 3 to image block position 13, ..., and transforming image block 16 to image block position 8, thereby obtaining the encrypted image.

[0038] In one embodiment, if the encrypted information includes the pixel position transformation relationship of each image block in the original image, then when performing encryption processing, the terminal device transforms the pixel positions in each image block based on the pixel position transformation relationship to obtain the encrypted image. Thus, the network device can determine the correspondence between the pixel positions of each image block in the original image and the pixel positions of each image block in the encrypted image through the encrypted information. Since the transformation from the original image to the encrypted image is merely a transformation of pixel positions, this encryption method is simple and can reduce computational overhead, latency, and power consumption.

[0039] For example, the encrypted information includes the transformation relationships of pixel positions in various image patches of the original image. This is equivalent to the encrypted information being a two-dimensional transformation set of pixels in a set of image patches, where each transformation corresponds to a patch, used to encode the arrangement order of pixels within that patch. In a typical ViT encoder, the original image is usually divided into a 14*14 patch grid, with each patch being 16*16 pixels in size. Therefore, this transformation set can be represented as a combined structure containing 14*14 16*16 grids. Figure 6 As shown, taking the original image as a 4*4 patch grid as an example, when the terminal device encrypts the image, it performs a corresponding temporal transformation operation on each patch in the original image according to the pixel position transformation relationship, thereby shuffling the arrangement order of pixels in each patch to obtain an encrypted image. This process effectively achieves finer-grained encryption processing while maintaining the overall image structure.

[0040] like Figure 7 As shown, the pixel position transformation described above can be represented by a two-dimensional grid. If the original image is divided into image blocks 1 to 16, and the pixels in image block 16 include pixels 1 to 9, where pixel 1 is at pixel position 1, pixel 2 is at pixel position 2, ..., and pixel 9 is at pixel position 9, then the pixel position transformation relationship represents transforming pixel 1 to pixel position 6, pixel 2 to pixel position 5, pixel 3 to pixel position 1, ..., and pixel 9 to pixel position 4, thereby obtaining the encrypted image.

[0041] In one embodiment, if the encrypted image includes the image block position transformation relationship of the original image and the pixel position transformation relationship of each image block in the original image, then when the terminal device performs encryption processing, it transforms the image blocks in the original image based on the image block position transformation relationship, and transforms the pixels in each image block based on the pixel position transformation relationship of each image block, thus obtaining the encrypted image. Therefore, the network device can determine the correspondence between the positions of the image blocks in the original image and the positions of the image blocks in the encrypted image, and the correspondence between the positions of the pixels in each image block in the original image and the positions of the pixels in each image block in the encrypted image, through the encrypted information. Since the transformation from the original image to the encrypted image is only a transformation process of image block positions and pixel positions, this encryption method is simple and can reduce computational overhead, latency, and power consumption.

[0042] For example: the encrypted information includes the position transformation relationship of image patches arranged in two dimensions of the original image, and the position transformation relationship of pixels within each image patch of the original image. In a typical ViT encoder, the original image is usually divided into a 14*14 patch grid, so the image patch position transformation relationship can be represented as a 14*14 position mapping used to control the arrangement order of patches in the input sequence. Each patch is 16*16 pixels in size. Figure 8 As shown, taking the original image as a 4*4 patch grid as an example, when the terminal device encrypts the image, it performs corresponding temporal transformation operations on the original image according to the image patch position transformation relationship, thereby scrambling the original spatial order of each patch in the original image. Then, according to the pixel position transformation relationship, it performs corresponding temporal transformation operations on each patch in the original image, thereby scrambling the arrangement order of pixels within each patch, thus obtaining the encrypted image.

[0043] like Figure 9As shown, both the image block position transformation and the pixel position transformation described above can be represented in the form of a two-dimensional grid. If the original image is divided into image blocks 1 to 16, image block 1 is located at image block position 1, image block 2 is located at image block position 2, ..., and image block 16 is located at image block position 16. The image block position transformation relationship means transforming image block 1 to image block position 15, image block 2 to image block position 7, image block 3 to image block position 13, ..., and image block 16 to image block position 8. Furthermore, the pixels in each image block (such as image block 16) include pixels 1 to 9, where pixel 1 is located at pixel position 1, pixel 2 at pixel position 2, ..., and pixel 9 at pixel position 9. The pixel position transformation relationship means transforming pixel 1 to pixel position 6, pixel 2 to pixel position 5, pixel 3 to pixel position 1, ..., and pixel 9 to pixel position 4, thereby obtaining the encrypted image.

[0044] The above embodiments employ a lightweight privacy protection method that is structurally compatible and requires no retraining. By performing patch-level and pixel-level encryption processing on the original image and designing corresponding image feature data restoration operations in the image processor, a unified approach to encryption protection of user data and model processing compatibility is effectively achieved, constructing a fully secure visual data processing mechanism that requires no plaintext restoration.

[0045] The following explains the pixel-level image processing procedure: In an optional specific embodiment, when the encryption information includes the pixel position transformation relationship in each image block of the original image, step 202 inputs the encrypted image into an image processing model to perform an image feature data restoration operation on the encrypted image based on the encryption information, thereby obtaining the image feature data corresponding to the encrypted image, including: The encrypted image is input into the image processing model, and the weight matrix of each convolutional kernel of the first convolutional layer is rearranged based on the pixel position transformation relationship corresponding to each image block to obtain the target convolutional kernel corresponding to each image block. Using the target convolution kernel corresponding to each image block, perform image block convolution processing on the image block to obtain the first image block embedding tensor; The first image block embedding tensor is fused with a preset position encoding tensor to obtain image feature data corresponding to the encrypted image. Based on the image feature data, a processing result corresponding to the task of the image processing model is obtained. The position encoding tensor and the first image block embedding tensor have the same dimension.

[0046] Specifically, if the encrypted information is the pixel position transformation relationship among the various image blocks of the original image, then the encrypted image is an image obtained by transforming the pixel positions among the various image blocks of the original image based on the pixel position transformation relationship, such as... Figure 10 As shown. After receiving the encrypted image and encrypted information, the network device inputs the encrypted image into the image processing model. The image processing model rearranges the weight matrices of each convolutional kernel in the first convolutional layer (two-dimensional convolutional layer) based on the pixel position transformation relationship corresponding to each image block, thus obtaining the target convolutional kernel corresponding to each image block. Specifically, for each image block, the rearrangement order of the weight matrices of each convolutional kernel in the first convolutional layer is based on the pixel position transformation relationship corresponding to the image block. For example, in an image block, if the pixel position transformation relationship means that pixel 1 changes from pixel position 1 to pixel position 6, and pixel 2 changes from pixel position 2 to pixel position 5, then in the weight matrix of each convolutional kernel, the weight corresponding to pixel 1 changes from weight position 1 to weight position 6, and the weight corresponding to pixel 2 changes from weight position 2 to weight position 5.

[0047] For each image patch, a patch convolution is performed using the target convolution kernel corresponding to that patch. This process yields the first image patch embedding tensor. Since convolution itself possesses weighted summation invariance, the convolution result after the rearrangement operation is consistent with the convolution result obtained from the original image input to the image processing model. Therefore, no additional adjustment or modification to the model structure is required. The patch embedding is then fused with a preset positional encoding tensor, forming a complete image feature representation. This results in the image feature data corresponding to the encrypted image, which is consistent with the image feature data obtained from the original image output to the image processing model. This image feature data is then fed into subsequent model layers for inference, yielding the processing result corresponding to the task of the image processing model.

[0048] Here, fusion processing refers to the element-wise superposition operation between two tensors.

[0049] like Figure 11 The following describes the specific workflow of pixel-level image processing: After receiving the encrypted image and encrypted information, the network device inputs the encrypted image into the image processing model. The model rearranges the weight matrices of the convolutional kernels in the first convolutional layer based on the pixel position transformations corresponding to each image patch, obtaining the target convolutional kernels for each image patch. Specifically, for each image patch, the weight matrices of the convolutional kernels in the first convolutional layer are rearranged according to the pixel position transformations corresponding to the image patch. For each image patch, the target convolutional kernel is used to perform patch convolution processing. This process yields the first image patch embedding tensor. Since convolution operations inherently possess weighted summation invariance, the convolution result after the rearrangement is consistent with the convolution result obtained from inputting the original image into the image processing model; therefore, no additional adjustments or modifications to the model structure are required.

[0050] In the above embodiments, the terminal device only needs to perform a simple temporal transformation of the image, resulting in minimal computational overhead and no significant latency or additional power consumption, making it suitable for resource-constrained mobile device environments. Furthermore, the network device does not need to reconstruct the original image; it only needs to reconstruct the image feature data within the image processing model, operating on the latent feature space (i.e., the latent space) rather than the original temporal domain of the image. Even if the intermediate results are leaked, the original image content cannot be directly reconstructed, thus effectively protecting user privacy. Moreover, the network device only needs to adjust the weight arrangement order of the two-dimensional convolutional layer according to the pixel position transformation relationship, without changing the values ​​themselves. Therefore, no fine-tuning or retraining is required, resulting in good model adaptability and deployment flexibility.

[0051] The following describes the Patch-level image processing procedure: In an optional specific embodiment, when the encryption information includes the image block position transformation relationship of the original image, step 202 inputs the encrypted image into an image processing model to perform an image feature data restoration operation on the encrypted image based on the encryption information, thereby obtaining the image feature data corresponding to the encrypted image, including: The encrypted image is input into the image processing model to perform image block convolution processing on the encrypted image through the second convolutional layer to obtain the second image block embedding tensor; Based on the image block position transformation relationship, the second image block embedding tensor is subjected to inverse transformation to obtain the third image block embedding tensor. The inverse transformation process indicates that the position transformation direction from the second image block embedding tensor to the third image block embedding tensor is opposite to the image block position transformation direction from the original image to the encrypted image. The third image block embedding tensor is fused with a preset position encoding tensor to obtain image feature data corresponding to the encrypted image. Based on the image feature data, a processing result corresponding to the task of the image processing model is obtained. The position encoding tensor and the third image block embedding tensor have the same dimension.

[0052] Specifically, if the encrypted information is the image patch position transformation relationship of the original image, then the encrypted image is an image obtained by transforming the image patches of the original image based on the image patch position transformation relationship, such as... Figure 12 As shown, after receiving the encrypted image and encrypted information, the network device inputs the encrypted image into the image processing model. The second convolutional layer of the image processing model performs image patch convolution on the encrypted image, obtaining the second image patch embedding tensor, commonly known as the patch token. The first and second convolutional layers can be the same or different layers. After the second image patch embedding tensor is generated, it undergoes an inverse transformation based on the image patch position transformation relationship to obtain the third image patch embedding tensor. For example, in the original image, the image patch position transformation relationship indicates that image patch 1 transforms from image patch position 1 to image patch position 15, and image patch 2 transforms from image patch position 1 to image patch position 7. Therefore, in the second image patch embedding tensor, the vector corresponding to image patch 1 transforms from vector position 1 to vector position 6, and the vector corresponding to image patch 2 transforms from vector position 2 to vector position 5.

[0053] Then, the embedding tensor of the third image block is fused with the preset positional encoding tensor. The two together constitute a complete image feature representation, which yields the image feature data corresponding to the encrypted image. This image feature data is consistent with the image feature data obtained by outputting the original image to the image processing model. This process does not require modification of the model's core structure, and subsequent computation flows remain completely consistent.

[0054] like Figure 13 The following describes the specific workflow of the above-mentioned Patch-level image processing: After receiving the encrypted image and encrypted information, the network device inputs the encrypted image into the image processing model. The second convolutional layer of the image processing model performs image block convolution processing on the encrypted image to obtain a second image block embedding tensor. After the second image block embedding tensor is generated, it undergoes an inverse transform based on the image block position transformation relationship to obtain a third image block embedding tensor. This third image block embedding tensor is then expanded and fused with a preset positional encoding tensor. Together, they constitute a complete image feature representation, i.e., the image feature data corresponding to the encrypted image. This image feature data is consistent with the image feature data obtained from the original image output to the image processing model.

[0055] In the above embodiments, the terminal device only needs to perform a simple temporal image transformation, resulting in minimal computational overhead and no significant latency or additional power consumption, making it suitable for resource-constrained mobile device environments. Furthermore, the network device does not need to reconstruct the original image; it only needs to reconstruct the image feature data within the image processing model, operating on the latent feature space (i.e., the latent space) rather than the original temporal domain of the image. Even if intermediate results are leaked, the original image content cannot be directly reconstructed, thus effectively protecting user privacy. Moreover, the network device only needs to perform a one-time rearrangement of the intermediate tensors based on the image block position transformation relationship, without modifying the model weights or structure. Therefore, no fine-tuning or retraining is required, demonstrating good versatility and adaptability.

[0056] The following explains the image processing procedures at the Patch and Pixel levels: In an optional specific embodiment, when the encryption information further includes the image block position transformation relationship of the original image, the step of fusing the first image block embedding tensor with a preset position encoding tensor to obtain image feature data corresponding to the encrypted image includes: Based on the image block position transformation relationship, the first image block embedding tensor is subjected to inverse transformation to obtain the fourth image block embedding tensor. The inverse transformation process indicates that the position transformation direction from the first image block embedding tensor to the fourth image block embedding tensor is opposite to the image block position transformation direction from the original image to the encrypted image. The fourth image block embedding tensor is fused with a preset position encoding tensor to obtain image feature data corresponding to the encrypted image. The position encoding tensor and the fourth image block embedding tensor have the same dimension.

[0057] Specifically, if the encrypted information is the pixel position transformation relationship between each image block of the original image and the image block position transformation relationship of the original image, then the encrypted image is an image obtained by transforming the pixel positions in each image block of the original image based on the pixel position transformation relationship and by transforming the image blocks of the original image, such as... Figure 14 As shown, after receiving the encrypted image and encrypted information, the network device inputs the encrypted image into the image processing model. The image processing model rearranges the weight matrices of each convolutional kernel in the first convolutional layer (a two-dimensional convolutional layer) based on the pixel position transformation relationships corresponding to each image patch, obtaining the target convolutional kernels for each image patch. Specifically, for each image patch, the rearrangement order of the weight matrices of each convolutional kernel in the first convolutional layer is based on the pixel position transformation relationships corresponding to the image patch. For each image patch, image patch convolution processing is performed using the target convolutional kernel corresponding to that image patch. This image patch convolution processing is performed on each image patch in the above manner to obtain the first image patch embedding tensor. Since the convolution operation itself possesses the invariance of weighted summation, after the above rearrangement operation, the convolution result is consistent with the convolution result obtained by inputting the original image into the image processing model; therefore, no additional adjustment or modification to the model structure is required.

[0058] Furthermore, based on the image patch position transformation relationship, the first image patch embedding tensor is inversely transformed to obtain the fourth image patch embedding tensor. Then, the fourth image patch embedding tensor is fused with the preset positional encoding tensor, and the two together constitute a complete image feature representation, i.e., the image feature data corresponding to the encrypted image. This image feature data is consistent with the image feature data obtained by outputting the original image to the image processing model. This processing does not require modification of the model's core structure, and subsequent calculation flows remain completely consistent.

[0059] like Figure 15 The following describes the specific workflow for Patch-level and Pixel-level image processing: After receiving the encrypted image and encrypted information, the network device inputs the encrypted image into the image processing model. The model rearranges the weight matrices of the convolutional kernels in the first convolutional layer based on the pixel position transformation relationships corresponding to each image block, obtaining the target convolutional kernels for each image block. For each image block, image block convolution processing is performed using the target convolutional kernels corresponding to that image block. This image block convolution processing is repeated to obtain the first image block embedding tensor. After generating the first image block embedding tensor, an inverse transformation is performed on it according to the image block position transformation relationships to obtain the fourth image block embedding tensor. Then, the fourth image block embedding tensor is fused with a preset position encoding tensor to obtain the image feature data corresponding to the encrypted image.

[0060] In the above embodiments, the terminal device only needs to perform simple image temporal transformations, resulting in minimal computational overhead and no significant latency or additional power consumption, making it suitable for resource-constrained mobile device environments. Furthermore, the network device does not need to reconstruct the original image; it only needs to reconstruct the image feature data within the image processing model, operating on the latent feature space (i.e., the latent space) rather than the original temporal domain of the image. Even if intermediate results are leaked, the original image content cannot be directly reconstructed, thus effectively protecting user privacy. Moreover, the network device only needs to adjust the weight arrangement order of the two-dimensional convolutional layer according to the pixel position transformation relationship, without changing its numerical values, and performs a one-time rearrangement operation on the order of intermediate tensors according to the image block position transformation relationship, without any fine-tuning or retraining, demonstrating good versatility and adaptability. Furthermore, the above hybrid encryption scheme introduces perturbations at both the patch and pixel levels, forming diverse encryption structures, effectively improving system security and anti-cracking capabilities, and further reducing the potential risk of data leakage.

[0061] In one embodiment, the solution of this application has strong versatility and scalability, and can be extended to the following aspects: 1. Compatible with multiple visual coding model structures The image processing model described above is not only applicable to the ViT architecture, but also to other mainstream visual coding models, including but not limited to: CNN-type encoders, such as ResNet, MobileNet, and ConvNeXt, can achieve pixel-level encryption alignment by adjusting the weight order of the convolution kernels; Hybrid structure models, such as Swing Transformer and CoAtNet, can also achieve decryption coupling by introducing an inverse mapping layer related to the Patch order in the token generation stage. Multimodal model front-ends, such as CLIP, BLIP, SigLIP, Flamingo, etc., mostly adopt standard ViT or CNN structures for their image input parts, and can also be directly compatible with the scheme in this application.

[0062] 2. Expand to multimodal data such as video, depth maps, and semantic maps. The encryption method in this application is not only applicable to the two-dimensional structure of image frames, but can also be extended to other spatial / temporal data structures, such as: Video data: Patch encryption or pixel encryption can be performed frame by frame in the video data, or time-dimensional shuffling can be introduced, such as three-dimensional patch rearrangement; Depth map / infrared image: Maintain the same data transformation logic as the original image (RGB image) above, and adapt to specific modalities; Intermediate representations such as semantic segmentation graphs or attention maps can be slightly perturbed before processing, so that they can both protect the intermediate results and not affect the downstream task processing.

[0063] 3. Supports diverse task scenarios This application is applicable to a variety of vision-based artificial intelligence tasks, especially systems that rely on the capabilities of large models for understanding, generation, or interaction, including but not limited to: Image classification (e.g., lane line recognition, vehicle license plate recognition), object detection, scene understanding; Image-text matching, image-text generation, and cross-modal question answering; Scenarios include intelligent customer service, human-computer dialogue, assisted driving, and real-time recognition of augmented reality (AR) / virtual reality (VR); Front-end sensing systems in smart wearable devices, such as smart glasses and head-mounted displays.

[0064] 4. Achieve local, private, and controllable computing by integrating with edge security modules. To further enhance the privacy protection capabilities of the method in a real-world system, the encryption steps can be deployed in the following modules: Trusted Execution Environment (TEE): such as ARM TrustZone and Intel SGX, where encryption algorithms are executed. Pre-processing modules on AI chips or Neural Processing Units (NPUs) enable encryption to be completed at the hardware level, increasing the cost of attacks. Image acquisition devices embed computing units, such as smart cameras and glasses front-end system-on-a-chip (SoC), integrating the solution of this application at the beginning of data acquisition.

[0065] 5. Complementary and integrated with cryptographic mechanisms It can be combined with the following methods to build a more complete security system: An identity authentication encryption method is added to the encryption method in the embodiments of this application to prevent encryption perturbation from being forged or replay attacks. Differential privacy injection perturbation: Image details are further scrambled before the original image is encrypted in this application to enhance security; Hash fingerprint generation: After obtaining the encrypted image using the scheme of this application, an irreversible digest is generated based on the hash fingerprint for consistency verification.

[0066] In summary, the embodiments of this application provide a lightweight, universal, and embeddable visual encryption-decryption integrated solution that significantly improves privacy and security in edge-cloud collaborative systems while ensuring model availability.

[0067] like Figure 16 As shown in the embodiments of this application, an image processing method is also provided, which may specifically include the following steps: Step 1601: Obtain the original image and the encrypted information for encrypting the original image; Step 1602: Transform the original image into an encrypted image according to the transformation relationship in the encrypted information; Step 1603: Send the encrypted image corresponding to the original image and the encrypted information.

[0068] Specifically, when the terminal device performs encryption, it transforms the original image into an encrypted image according to the transformation relationship in the encryption information. This encryption process effectively encrypts the original image while maintaining the overall image structure. The terminal device packages the encryption information and the encrypted image together and uploads them to the network device. The network device receives the encrypted image of the original image and the encryption information sent by the terminal device.

[0069] The original image can also be extended to multimodal data such as video, depth map, and semantic map.

[0070] In one embodiment, after the network device performs image processing to obtain the processing result, if the processing result needs to be encrypted, the network device uses the processing result as the original image and encrypts it according to steps 1601 to 1602, and then transmits the encrypted image and encrypted information to the terminal device.

[0071] Optionally, step 1602, which transforms the original image into an encrypted image according to the transformation relationship in the encrypted information, includes: When the encrypted information includes the image block position transformation relationship of the original image, the image blocks of the original image are transformed according to the image block position transformation relationship to obtain the encrypted image; and / or, When the encrypted information includes the pixel position transformation relationship in each image block of the original image, the pixel positions in each image block of the original image are transformed according to the pixel position transformation relationship to obtain the encrypted image.

[0072] In one embodiment, if the encrypted information includes the image block position transformation relationship of the original image, the terminal device performs position transformation on the image blocks in the original image based on the image block position transformation relationship during encryption processing to obtain the encrypted image. Since the process from the original image to the encrypted image is only a transformation of the image block positions, this encryption method is simple and can reduce computational overhead, latency, and power consumption.

[0073] For example, the encrypted information includes a two-dimensional arrangement of image patch positions in the original image, which is used to encode the positional order of the individual image patches in the original image. In a typical ViT encoder, the original image is usually divided into a 14x14 patch grid; therefore, the image patch position transformation can be represented as a 14x14 position mapping to control the arrangement order of patches in the input sequence. Figure 4 As shown, taking the original image as a 4*4 patch grid as an example, when the terminal device encrypts the image, it performs a corresponding temporal transformation operation on the original image according to the image patch position transformation relationship, thereby scrambling the original spatial order of each patch in the original image to obtain an encrypted image. This process is essentially rearranging the structure of the original image at the patch level to achieve effective encryption of the original image content while maintaining the compatibility of the visual encoder.

[0074] like Figure 5 As shown, the above image block position transformation can be represented by a two-dimensional grid. If the original image is divided into image blocks 1 to 16, image block 1 is located at image block position 1, image block 2 is located at image block position 2, ..., and image block 16 is located at image block position 16. The image block position transformation relationship means transforming image block 1 to image block position 15, transforming image block 2 to image block position 7, transforming image block 3 to image block position 13, ..., and transforming image block 16 to image block position 8, thereby obtaining the encrypted image.

[0075] In one embodiment, if the encrypted information includes the pixel position transformation relationship in each image block of the original image, then when performing encryption processing, the terminal device transforms the pixel positions in each image block based on the pixel position transformation relationship in each image block to obtain the encrypted image. Since the process of transforming the original image into the encrypted image is only a transformation of pixel positions, this encryption method is simple and can reduce computational overhead, latency, and power consumption.

[0076] For example, the encrypted information includes the transformation relationships of pixel positions in various image patches of the original image. This is equivalent to the encrypted information being a two-dimensional transformation set of pixels in a set of image patches, where each transformation corresponds to a patch, used to encode the arrangement order of pixels within that patch. In a typical ViT encoder, the original image is usually divided into a 14*14 patch grid, with each patch being 16*16 pixels in size. Therefore, this transformation set can be represented as a combined structure containing 14*14 16*16 grids. Figure 6 As shown, taking the original image as a 4*4 patch grid as an example, when the terminal device encrypts the image, it performs a corresponding temporal transformation operation on each patch in the original image according to the pixel position transformation relationship, thereby shuffling the arrangement order of pixels in each patch to obtain an encrypted image. This process effectively achieves finer-grained encryption processing while maintaining the overall image structure.

[0077] like Figure 7 As shown, the pixel position transformation described above can be represented by a two-dimensional grid. If the original image is divided into image blocks 1 to 16, and the pixels in image block 16 include pixels 1 to 9, where pixel 1 is at pixel position 1, pixel 2 is at pixel position 2, ..., and pixel 9 is at pixel position 9, then the pixel position transformation relationship represents transforming pixel 1 to pixel position 6, pixel 2 to pixel position 5, pixel 3 to pixel position 1, ..., and pixel 9 to pixel position 4, thereby obtaining the encrypted image.

[0078] In one embodiment, if the encrypted image includes the image block position transformation relationship of the original image and the pixel position transformation relationship of each image block in the original image, then when the terminal device performs encryption processing, it transforms the image blocks in the original image based on the image block position transformation relationship, and transforms the pixels in each image block based on the pixel position transformation relationship of each image block, thus obtaining the encrypted image. Therefore, the network device can determine the correspondence between the positions of the image blocks in the original image and the positions of the image blocks in the encrypted image, and the correspondence between the positions of the pixels in each image block in the original image and the positions of the pixels in each image block in the encrypted image, through the encrypted information. Since the transformation from the original image to the encrypted image is only a transformation process of image block positions and pixel positions, this encryption method is simple and can reduce computational overhead, latency, and power consumption.

[0079] For example: the encrypted information includes the position transformation relationship of image patches arranged in two dimensions of the original image, and the position transformation relationship of pixels within each image patch of the original image. In a typical ViT encoder, the original image is usually divided into a 14*14 patch grid, so the image patch position transformation relationship can be represented as a 14*14 position mapping used to control the arrangement order of patches in the input sequence. Each patch is 16*16 pixels in size. Figure 8 As shown, taking the original image as a 4*4 patch grid as an example, when the terminal device encrypts the image, it performs corresponding temporal transformation operations on the original image according to the image patch position transformation relationship, thereby scrambling the original spatial order of each patch in the original image. Then, according to the pixel position transformation relationship, it performs corresponding temporal transformation operations on each patch in the original image, thereby scrambling the arrangement order of pixels within each patch, thus obtaining the encrypted image.

[0080] like Figure 9As shown, both the image block position transformation and the pixel position transformation described above can be represented in the form of a two-dimensional grid. If the original image is divided into image blocks 1 to 16, image block 1 is located at image block position 1, image block 2 is located at image block position 2, ..., and image block 16 is located at image block position 16. The image block position transformation relationship means transforming image block 1 to image block position 15, image block 2 to image block position 7, image block 3 to image block position 13, ..., and image block 16 to image block position 8. Furthermore, the pixels in each image block (such as image block 16) include pixels 1 to 9, where pixel 1 is located at pixel position 1, pixel 2 at pixel position 2, ..., and pixel 9 at pixel position 9. The pixel position transformation relationship means transforming pixel 1 to pixel position 6, pixel 2 to pixel position 5, pixel 3 to pixel position 1, ..., and pixel 9 to pixel position 4, thereby obtaining the encrypted image.

[0081] The above embodiments employ a lightweight privacy protection method that is structurally compatible and requires no retraining. By performing patch-level and pixel-level encryption processing on the original image and designing corresponding image feature data restoration operations in the image processor, a unified approach to encryption protection of user data and model processing compatibility is effectively achieved, constructing a fully secure visual data processing mechanism that requires no plaintext restoration.

[0082] The image processing method in Embodiment 1 provided in this application can be executed by an image processing device. This application uses an image processing device executing the image processing method in Embodiment 1 as an example to illustrate the image processing device provided in this application.

[0083] like Figure 17 As shown in the figure, this application embodiment also provides an image processing apparatus 1700, specifically including: The receiving module 1701 is used to receive the encrypted image corresponding to the original image and the encryption information, wherein the encryption information is the transformation relationship used to transform the original image into the encrypted image; The restoration module 1702 is used to input the encrypted image into the image processing model to perform an image feature data restoration operation on the encrypted image based on the encryption information, so as to obtain the image feature data corresponding to the encrypted image. The image feature data corresponding to the encrypted image is the same as the image feature data corresponding to the original image.

[0084] Optionally, if the encrypted information includes the image patch position transformation relationship of the original image, the encrypted image is an image obtained by transforming the image patches of the original image based on the image patch position transformation relationship; and / or, When the encrypted information includes the pixel position transformation relationship in each image block of the original image, the encrypted image is an image obtained by transforming the pixel positions in each image block of the original image based on the pixel position transformation relationship.

[0085] Optionally, when the encrypted information includes the pixel position transformation relationship in each image block of the original image, the restoration module 1702 is specifically used for: The encrypted image is input into the image processing model, and the weight matrix of each convolutional kernel of the first convolutional layer is rearranged based on the pixel position transformation relationship corresponding to each image block to obtain the target convolutional kernel corresponding to each image block. Using the target convolution kernel corresponding to each image block, perform image block convolution processing on the image block to obtain the first image block embedding tensor; The first image block embedding tensor is fused with a preset position encoding tensor to obtain image feature data corresponding to the encrypted image. Based on the image feature data, a processing result corresponding to the task of the image processing model is obtained. The position encoding tensor and the first image block embedding tensor have the same dimension.

[0086] Optionally, when the encrypted information includes the image block position transformation relationship of the original image, the restoration module 1702 is specifically used for: The encrypted image is input into the image processing model to perform image block convolution processing on the encrypted image through the second convolutional layer to obtain the second image block embedding tensor; Based on the image block position transformation relationship, the second image block embedding tensor is subjected to inverse transformation to obtain the third image block embedding tensor. The inverse transformation process indicates that the position transformation direction from the second image block embedding tensor to the third image block embedding tensor is opposite to the image block position transformation direction from the original image to the encrypted image. The third image block embedding tensor is fused with a preset position encoding tensor to obtain image feature data corresponding to the encrypted image. Based on the image feature data, a processing result corresponding to the task of the image processing model is obtained. The position encoding tensor and the third image block embedding tensor have the same dimension.

[0087] Optionally, when the encrypted information further includes the image block position transformation relationship of the original image, the restoration module 1702, when fusing the first image block embedding tensor with a preset position encoding tensor to obtain the image feature data corresponding to the encrypted image, is specifically used for: Based on the image block position transformation relationship, the first image block embedding tensor is subjected to inverse transformation to obtain the fourth image block embedding tensor. The inverse transformation process indicates that the position transformation direction from the first image block embedding tensor to the fourth image block embedding tensor is opposite to the image block position transformation direction from the original image to the encrypted image. The fourth image block embedding tensor is fused with a preset position encoding tensor to obtain image feature data corresponding to the encrypted image. The position encoding tensor and the fourth image block embedding tensor have the same dimension.

[0088] The image processing apparatus provided in this application embodiment can achieve... Figures 1 to 15 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0089] The image processing method in Embodiment 2 provided in this application can be executed by an image processing device. This application uses an image processing device executing the image processing method in Embodiment 2 as an example to illustrate the image processing device provided in this application.

[0090] like Figure 18 As shown, this application embodiment also provides an image processing apparatus 1800, specifically including: The acquisition module 1801 is used to acquire the original image and the encrypted information for encrypting the original image; The transformation module 1802 is used to transform the original image into an encrypted image according to the transformation relationship in the encrypted information; The sending module 1803 is used to send the encrypted image corresponding to the original image and the encrypted information to the network device.

[0091] Optionally, when the transformation module 1802 transforms the original image into an encrypted image according to the transformation relationship in the encrypted information, it is specifically used for: When the encrypted information includes the image block position transformation relationship of the original image, the image blocks of the original image are transformed according to the image block position transformation relationship to obtain the encrypted image; and / or, When the encrypted information includes the pixel position transformation relationship in each image block of the original image, the pixel positions in each image block of the original image are transformed according to the pixel position transformation relationship to obtain the encrypted image.

[0092] The image processing apparatus provided in this application embodiment can achieve... Figures 4 to 9 , Figure 16 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0093] The image processing device in the above embodiments can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This application does not specifically limit the scope of the embodiments.

[0094] The image processing device in the above embodiments can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this application does not specifically limit the specific operating system used.

[0095] Optionally, such as Figure 19 As shown, this application embodiment also provides an electronic device 1900, including a processor 1901 and a memory 1902. The memory 1902 stores a program or instructions that can run on the processor 1901. When the program or instructions are executed by the processor 1901, they implement the various steps of the above-described image processing method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0096] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0097] Figure 20 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0098] The electronic device 1000 includes, but is not limited to, components such as: radio frequency unit 1001, network module 1002, audio output unit 1003, input unit 1004, sensor 1005, display unit 1006, user input unit 1007, interface unit 1008, memory 1009, and processor 1010.

[0099] Those skilled in the art will understand that the electronic device 1000 may also include a power supply (such as a battery) for supplying power to various components. The power supply may be logically connected to the processor 1010 through a power management system, thereby enabling functions such as managing charging, discharging, and power consumption through the power management system. Figure 20 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0100] In one embodiment, the input unit 1004 is used to receive an encrypted image corresponding to the original image and encryption information, wherein the encryption information is the transformation relationship used to transform the original image into the encrypted image; The processor 1010 is used to input the encrypted image into an image processing model to perform an image feature data restoration operation on the encrypted image based on the encryption information, thereby obtaining the image feature data corresponding to the encrypted image. The image feature data corresponding to the encrypted image is the same as the image feature data corresponding to the original image.

[0101] Optionally, if the encrypted information includes the image patch position transformation relationship of the original image, the encrypted image is an image obtained by transforming the image patches of the original image based on the image patch position transformation relationship; and / or, When the encrypted information includes the pixel position transformation relationship in each image block of the original image, the encrypted image is an image obtained by transforming the pixel positions in each image block of the original image based on the pixel position transformation relationship.

[0102] Optionally, when the encrypted information includes the pixel position transformation relationship in each image block of the original image, the processor 1010 is specifically used for: The encrypted image is input into the image processing model, and the weight matrix of each convolutional kernel of the first convolutional layer is rearranged based on the pixel position transformation relationship corresponding to each image block to obtain the target convolutional kernel corresponding to each image block. Using the target convolution kernel corresponding to each image block, perform image block convolution processing on the image block to obtain the first image block embedding tensor; The first image block embedding tensor is fused with a preset position encoding tensor to obtain image feature data corresponding to the encrypted image. Based on the image feature data, a processing result corresponding to the task of the image processing model is obtained. The position encoding tensor and the first image block embedding tensor have the same dimension.

[0103] Optionally, when the encrypted information includes the image patch position transformation relationship of the original image, the processor 1010 is specifically used for: The encrypted image is input into the image processing model to perform image block convolution processing on the encrypted image through the second convolutional layer to obtain the second image block embedding tensor; Based on the image block position transformation relationship, the second image block embedding tensor is subjected to inverse transformation to obtain the third image block embedding tensor. The inverse transformation process indicates that the position transformation direction from the second image block embedding tensor to the third image block embedding tensor is opposite to the image block position transformation direction from the original image to the encrypted image. The third image block embedding tensor is fused with a preset position encoding tensor to obtain image feature data corresponding to the encrypted image. Based on the image feature data, a processing result corresponding to the task of the image processing model is obtained. The position encoding tensor and the third image block embedding tensor have the same dimension.

[0104] Optionally, when the encrypted information further includes the image patch position transformation relationship of the original image, the processor 1010, when fusing the first image patch embedding tensor with a preset position encoding tensor to obtain the image feature data corresponding to the encrypted image, specifically performs the following: Based on the image block position transformation relationship, the first image block embedding tensor is subjected to inverse transformation to obtain the fourth image block embedding tensor. The inverse transformation process indicates that the position transformation direction from the first image block embedding tensor to the fourth image block embedding tensor is opposite to the image block position transformation direction from the original image to the encrypted image. The fourth image block embedding tensor is fused with a preset position encoding tensor to obtain image feature data corresponding to the encrypted image. The position encoding tensor and the fourth image block embedding tensor have the same dimension.

[0105] In another embodiment, the processor 1010 is configured to acquire the original image and encrypted information for encrypting the original image; The original image is transformed into an encrypted image according to the transformation relationship in the encrypted information; Send the encrypted image corresponding to the original image, along with the encrypted information.

[0106] Optionally, when the processor 1010 transforms the original image into an encrypted image according to the transformation relationship in the encrypted information, it specifically performs the following: When the encrypted information includes the image block position transformation relationship of the original image, the image blocks of the original image are transformed according to the image block position transformation relationship to obtain the encrypted image; and / or, When the encrypted information includes the pixel position transformation relationship in each image block of the original image, the pixel positions in each image block of the original image are transformed according to the pixel position transformation relationship to obtain the encrypted image.

[0107] It should be understood that, in this embodiment, the input unit 1004 may include a graphics processing unit (GPU) 10041 and a microphone 10042. The GPU 10041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1006 may include a display panel 10061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, etc. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072. The touch panel 10071 is also called a touch screen. The touch panel 10071 may include a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, joysticks, etc., which will not be described in detail here.

[0108] The memory 1009 can be used to store software programs and various data. The memory 1009 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1009 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1009 in this embodiment includes, but is not limited to, these and any other suitable types of memory.

[0109] The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into the processor 1010.

[0110] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described image processing method embodiments and achieve the same technical effects. To avoid repetition, they will not be described again here.

[0111] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0112] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described image processing method embodiments and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0113] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0114] This application provides a computer program product, which is stored in a storage medium and executed by at least one processor to implement the various processes of the above-described image processing method embodiments, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0115] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element. Furthermore, it should be noted that the scope of the methods and apparatuses in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0116] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0117] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

Claims

1. An image processing method, characterized in that, include: Receive the encrypted image corresponding to the original image and the encryption information, wherein the encryption information is the transformation relationship used to transform the original image into the encrypted image; The encrypted image is input into an image processing model to perform an image feature data restoration operation on the encrypted image based on the encryption information, thereby obtaining the image feature data corresponding to the encrypted image. The image feature data corresponding to the encrypted image is the same as the image feature data corresponding to the original image.

2. The method according to claim 1, characterized in that, When the encrypted information includes the image block position transformation relationship of the original image, the encrypted image is an image obtained by transforming the image blocks of the original image based on the image block position transformation relationship; And / or, When the encrypted information includes the pixel position transformation relationship in each image block of the original image, the encrypted image is an image obtained by transforming the pixel positions in each image block of the original image based on the pixel position transformation relationship.

3. The method according to claim 2, characterized in that, When the encryption information includes the pixel position transformation relationship in each image block of the original image, the step of inputting the encrypted image into an image processing model to perform an image feature data restoration operation on the encrypted image based on the encryption information to obtain the image feature data corresponding to the encrypted image includes: The encrypted image is input into the image processing model, and the weight matrix of each convolutional kernel of the first convolutional layer is rearranged based on the pixel position transformation relationship corresponding to each image block to obtain the target convolutional kernel corresponding to each image block. Using the target convolution kernel corresponding to each image block, perform image block convolution processing on the image block to obtain the first image block embedding tensor; The first image block embedding tensor is fused with a preset position encoding tensor to obtain image feature data corresponding to the encrypted image. Based on the image feature data, a processing result corresponding to the task of the image processing model is obtained. The position encoding tensor and the first image block embedding tensor have the same dimension.

4. The method according to claim 2, characterized in that, When the encrypted information includes the image patch position transformation relationship of the original image, the step of inputting the encrypted image into the image processing model to perform an image feature data restoration operation on the encrypted image based on the encrypted information to obtain the image feature data corresponding to the encrypted image includes: The encrypted image is input into the image processing model to perform image block convolution processing on the encrypted image through the second convolutional layer to obtain the second image block embedding tensor; Based on the image block position transformation relationship, the second image block embedding tensor is subjected to inverse transformation to obtain the third image block embedding tensor. The inverse transformation process indicates that the position transformation direction from the second image block embedding tensor to the third image block embedding tensor is opposite to the image block position transformation direction from the original image to the encrypted image. The third image block embedding tensor is fused with a preset position encoding tensor to obtain image feature data corresponding to the encrypted image. Based on the image feature data, a processing result corresponding to the task of the image processing model is obtained. The position encoding tensor and the third image block embedding tensor have the same dimension.

5. The method according to claim 3, characterized in that, When the encrypted information also includes the image block position transformation relationship of the original image, the step of fusing the first image block embedding tensor with a preset position encoding tensor to obtain the image feature data corresponding to the encrypted image includes: Based on the image block position transformation relationship, the first image block embedding tensor is subjected to inverse transformation to obtain the fourth image block embedding tensor. The inverse transformation process indicates that the position transformation direction from the first image block embedding tensor to the fourth image block embedding tensor is opposite to the image block position transformation direction from the original image to the encrypted image. The fourth image block embedding tensor is fused with a preset position encoding tensor to obtain image feature data corresponding to the encrypted image. The position encoding tensor and the fourth image block embedding tensor have the same dimension.

6. An image processing method, characterized in that, include: Acquire the original image and the encrypted information that encrypts the original image; The original image is transformed into an encrypted image according to the transformation relationship in the encrypted information; Send the encrypted image corresponding to the original image, along with the encrypted information.

7. The method according to claim 6, characterized in that, The step of transforming the original image into an encrypted image according to the transformation relationship in the encrypted information includes: When the encrypted information includes the image block position transformation relationship of the original image, the image blocks of the original image are transformed according to the image block position transformation relationship to obtain the encrypted image; and / or, When the encrypted information includes the pixel position transformation relationship in each image block of the original image, the pixel positions in each image block of the original image are transformed according to the pixel position transformation relationship to obtain the encrypted image.

8. An image processing apparatus, characterized in that, include: A receiving module is used to receive an encrypted image corresponding to the original image and encrypted information, wherein the encrypted information is the transformation relationship used to transform the original image into the encrypted image; The restoration module is used to input the encrypted image into the image processing model to perform an image feature data restoration operation on the encrypted image based on the encryption information, so as to obtain the image feature data corresponding to the encrypted image. The image feature data corresponding to the encrypted image is the same as the image feature data corresponding to the original image.

9. An image processing apparatus, characterized in that, include: The acquisition module is used to acquire the original image and the encrypted information that encrypts the original image; A transformation module is used to transform the original image into an encrypted image according to the transformation relationship in the encrypted information; The sending module is used to send the encrypted image corresponding to the original image and the encrypted information to the network device.

10. An electronic device, characterized in that, It includes a processor and a memory, the memory storing a program or instructions that can run on the processor, the program or instructions being executed by the processor to implement the steps of the image processing method as claimed in any one of claims 1-5 or to implement the steps of the image processing method as claimed in any one of claims 6-7.

11. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the image processing method as described in any one of claims 1-5 or as described in any one of claims 6-7.

Images