Method, apparatus and image processing system for image data augmentation

Abstract

Embodiments of the present specification provide a method, device and image processing system for image data enhancement. In the method for image data enhancement, a sequence of image data enhancement instructions to be filled is obtained from a non-secure computing environment, wherein the image data enhancement instruction to be filled comprises a type of image data enhancement operation and a to-be-filled field for indicating a size of an image to be processed; for each image data enhancement instruction to be filled, the to-be-filled field of the image data enhancement instruction to be filled is filled in a secure computing environment according to corresponding image description information, to obtain a complete image data enhancement instruction, wherein the image description information is obtained according to a result of decrypting a ciphertext image from the non-secure computing environment, and the image description information comprises size information; and a set of processed images is obtained by performing data enhancement processing on corresponding images to be processed according to each obtained image data enhancement instruction.

Description

Technical Field

[0001] The embodiments described herein generally relate to the field of computer technology, and more particularly to methods, apparatus and image processing systems for image data enhancement. Background Technology

[0002] With the rapid development of artificial intelligence technology and the increasing emphasis on data security and privacy protection, effectively training machine learning models while protecting data security and privacy has become a research hotspot. Existing technologies have proposed solutions for deep learning training or inference in secure multi-party computation (MPC) environments based on chips such as FPGAs (Field Programmable Gate Arrays) or ASICs (Application Specific Integrated Circuits). Data augmentation computation is a crucial component of the deep learning training framework. Therefore, for deep learning systems involving vision, achieving image data augmentation within a privacy-preserving secure computing environment such as an FPGA or ASIC has become a key challenge. Summary of the Invention

[0003] In view of the above, embodiments of this specification provide a method, apparatus, and image processing system for image data enhancement. Using this method, apparatus, and system, an image data enhancement effect in a secure computing environment can be achieved by filling in an image data enhancement instruction sequence from an insecure computing environment within a secure computing environment.

[0004] According to one aspect of an embodiment of this specification, a method for image data augmentation is provided, comprising: acquiring a sequence of image data augmentation instructions to be filled from an insecure computing environment, wherein each image data augmentation instruction to be filled includes a type of image data augmentation operation and a field to be filled, the field to be filled including a field for indicating the size of the image to be processed; performing the following image data augmentation operation in a secure computing environment: for each image data augmentation instruction to be filled, filling the field to be filled of the image data augmentation instruction to be filled according to the acquired image description information of the image to be processed corresponding to the image data augmentation instruction to be filled, to obtain a fully filled image data augmentation instruction, wherein the image description information of the image to be processed is obtained based on the result of decrypting an encrypted image from the insecure computing environment, the image description information including size information; and performing data augmentation processing on the corresponding image to be processed according to the obtained image data augmentation instructions to obtain a processed image set.

[0005] According to another aspect of the embodiments of this specification, an apparatus for image data enhancement is provided, comprising: a to-be-filled instruction acquisition unit configured to acquire a sequence of to-be-filled image data enhancement instructions from an insecure computing environment, wherein each to-be-filled image data enhancement instruction includes a type of image data enhancement operation and a field to be filled, the field to be filled including a field for indicating the size of the image to be processed; an instruction filling unit configured, in a secure computing environment, for each to-be-filled image data enhancement instruction, to fill the field to be filled of the to-be-filled image data enhancement instruction according to the acquired image description information of the image to be processed corresponding to the to-be-filled image data enhancement instruction, to obtain a fully filled image data enhancement instruction, wherein the image description information of the image to be processed is obtained based on the result of decrypting an encrypted image from the insecure computing environment, the image description information including size information; and a data enhancement unit configured, in a secure computing environment, to perform data enhancement processing on the corresponding image to be processed according to the acquired image data enhancement instructions, to obtain a processed image set.

[0006] According to another aspect of the embodiments of this specification, an image processing system is provided, comprising: an image decryption device configured to decrypt an encrypted image from an insecure computing environment to obtain an image to be processed and corresponding image description information; an image data enhancement device as described above; a deep learning device configured to perform calculations on the processed image using a deep learning operator module to obtain an image calculation result; and a communication device.

[0007] According to another aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program, which, when executed by a processor, implements the method for image data enhancement as described above.

[0008] According to another aspect of the embodiments of this specification, a computer program product is provided, including a computer program that is executed by a processor to implement the method for image data enhancement as described above. Attached Figure Description

[0009] A further understanding of the nature and advantages of this specification can be achieved by referring to the following figures. In the figures, similar components or features may have the same reference numerals.

[0010] Figure 1 An exemplary architecture of a method, apparatus, and image processing system for image data enhancement according to embodiments of this specification is shown.

[0011] Figure 2A flowchart illustrating an example of a method for image data enhancement according to an embodiment of this specification is shown.

[0012] Figure 3 A schematic diagram of an example chip for performing an image data enhancement processing procedure according to an embodiment of this specification is shown.

[0013] Figure 4 A flowchart illustrating an example of a data augmentation process according to an embodiment of this specification is shown.

[0014] Figure 5 A schematic diagram of yet another example of a chip for performing an image data enhancement process according to an embodiment of this specification is shown.

[0015] Figure 6 A schematic diagram illustrates an example of an image data augmentation instruction to be filled according to an embodiment of this specification.

[0016] Figure 7 A schematic diagram illustrates an example of an image data augmentation instruction for filling in completeness according to an embodiment of this specification.

[0017] Figure 8 A schematic diagram of yet another example of a chip for performing an image data enhancement method according to an embodiment of this specification is shown.

[0018] Figure 9 A schematic diagram of an example system-on-a-chip for performing an image data enhancement process according to an embodiment of this specification is shown.

[0019] Figure 10 A block diagram illustrating an example of an apparatus for image data enhancement according to an embodiment of this specification is shown.

[0020] Figure 11 A block diagram illustrating yet another example of an apparatus for image data enhancement according to embodiments of this specification is shown.

[0021] Figure 12 A block diagram illustrating yet another example of an apparatus for image data enhancement according to an embodiment of this specification is shown.

[0022] Figure 13 A block diagram of an example image processing system according to an embodiment of this specification is shown.

[0023] Figure 14 A block diagram illustrating an example of an image decryption apparatus in an image processing system according to an embodiment of this specification is shown.

[0024] Figure 15A block diagram illustrating an example of an apparatus for image data enhancement according to an embodiment of this specification is shown. Detailed Implementation

[0025] The subject matter described herein will be discussed below with reference to exemplary embodiments. It should be understood that these embodiments are discussed merely to enable those skilled in the art to better understand and implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. The function and arrangement of the elements discussed may be changed without departing from the scope of the embodiments described herein. Various processes or components may be omitted, substituted, or added as needed in the various examples. Furthermore, features described in some examples may be combined in other examples.

[0026] As used herein, the term "comprising" and its variations are open terms meaning "including but not limited to". The term "based on" means "at least partially based on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other definitions, whether explicit or implicit, may be included below. Unless explicitly indicated by the context, the definition of a term shall remain consistent throughout the specification.

[0027] The method, apparatus, and image processing system for image data enhancement according to embodiments of this specification will now be described in detail with reference to the accompanying drawings.

[0028] Figure 1 An exemplary architecture 100 for a method, apparatus, and image processing system for image data enhancement according to embodiments of this specification is shown.

[0029] exist Figure 1 In this context, the network can be used to interconnect a host 111 in an insecure computing environment 110 and an image data enhancement device 122 and an image post-processing device 123 in a secure computing environment 120.

[0030] A network can be any type of network capable of interconnecting network entities. A network can be a single network or a combination of various networks. In terms of coverage, a network can be a Local Area Network (LAN), a Wide Area Network (WAN), etc. In terms of the carrying medium, a network can be a wired network, a wireless network, etc. In terms of data switching technology, a network can be a circuit-switched network, a packet-switched network, etc.

[0031] The host computer 111, the image data enhancement device 122, and the image post-processing device 123 can be any type of electronic computing device capable of connecting to a network, accessing servers or websites on the network, and processing data or signals. For example, the host computer 111, the image data enhancement device 122, and the image post-processing device 123 can be a desktop computer, a laptop computer, a tablet computer, a smartphone, etc. Optionally, the image data enhancement device 122 and the image post-processing device 123 can also be based on chips such as FPGAs or ASICs. Although in Figure 1 Only one host 111, one device 122 for image data enhancement, and one image post-processing device 123 are shown in the diagram. However, it should be understood that different numbers of hosts 111, devices 122 for image data enhancement, and image post-processing devices 123 may be connected to the network described above.

[0032] In one implementation, host 111 can be used by a user. Host 111 may include application clients that can provide various services to the user. In one example, the application client may be a programming application, a machine learning application, etc. In some cases, host 111 may transmit user-input messages to communication device 121 and receive responses associated with the aforementioned messages from communication device 121. In this document, "message" may refer to any input information, such as a sequence of image data augmentation instructions to be filled from user input.

[0033] The image data augmentation apparatus 122 can receive a sequence of image data augmentation instructions from an insecure computing environment via the communication device 121. In one example, the image data augmentation apparatus 122 can perform an image data augmentation operation according to the received sequence of image data augmentation instructions to obtain an image with augmented data. Optionally, the image data augmentation apparatus 122 can return the obtained image with augmented data to the host 111 via the communication device 121. Optionally, the image data augmentation apparatus 122 can also send the obtained image with augmented data to the image post-processing apparatus 123 for further processing (e.g., obtaining image calculation results using a deep learning operator module).

[0034] It should be understood that Figure 1 All network entities shown are exemplary, and any other network entities may be involved in Architecture 100 depending on the specific application requirements.

[0035] Figure 2 A flowchart of a method 200 for image data enhancement according to an embodiment of this specification is shown.

[0036] like Figure 2As shown, at 210, the sequence of image data augmentation instructions to be filled is obtained from the insecure computing environment.

[0037] In this embodiment, a sequence of image data augmentation instructions to be filled can be obtained from an insecure computing environment. Each of these instructions may include the type of image data augmentation operation and a field to be filled. The field to be filled may include a field indicating the size of the image to be processed.

[0038] In this embodiment, the type of image data enhancement operation can be used to indicate different image data enhancement operations, such as cropping, flipping, resizing, and normalizing. In privacy-preserving scenarios, sensitive information of the original data (e.g., image data dimensions) often cannot be exposed in insecure computing environments. Image data enhancement instructions used to indicate image data enhancement operations often need to know relevant information about the image to be processed, such as the storage location and dimensions of the image. Therefore, each image data enhancement instruction in the sequence of instructions to be filled in an insecure computing environment may contain a field indicating the dimensions of the image to be processed, which needs to be filled in subsequent stages.

[0039] It should be noted that the above sequence of image data enhancement instructions can be used to instruct a series of image data enhancement processes on the original image to be processed. For example, it can first be flipped, and then cropped. Each image data enhancement instruction in the above sequence can correspond to its own image to be processed. For example, the image to be processed corresponding to the flip instruction can be the original image to be processed. The image to be processed corresponding to the crop instruction can be the original image to be processed after being flipped.

[0040] At 220, perform the image data augmentation operations described in steps 221 and 222 in a secure computing environment.

[0041] In this embodiment, the image data enhancement operations described in steps 221 and 222 can be performed in a secure computing environment. In one example, the secure computing environment can be the internal secure environment of one of the participants in a multi-party secure computing environment. In another example, the secure computing environment can also be a trusted execution environment (TEE).

[0042] In step 221, for each image data augmentation instruction to be filled, the field to be filled of the image data augmentation instruction is filled according to the obtained image description information of the image to be processed corresponding to the image data augmentation instruction to be filled, so as to obtain a fully filled image data augmentation instruction.

[0043] In this embodiment, the image description information of the image to be processed can be obtained from the result of decrypting the ciphertext image from an insecure computing environment. The image description information may include size information. In one example, the size information can be represented by the number of pixels in the width and height of the image.

[0044] In one example, an encrypted image from an insecure computing environment can first be decrypted to obtain a decrypted image and its corresponding image description information. The decrypted image can be the original image to be processed. The image description information can include the size information of the original image to be processed. Then, based on the image description information corresponding to the decrypted image, the fields to be filled in each image data augmentation instruction can be filled to obtain a fully filled image data augmentation instruction. Optionally, the image description information can also include image type information, such as a grayscale image or a color image.

[0045] As mentioned earlier, each image data enhancement instruction in the aforementioned sequence of image data enhancement instructions can correspond to its own image to be processed. Therefore, the image description information of the image to be processed corresponding to each image data enhancement instruction can also be obtained based on the decrypted image and the instructions preceding the image data enhancement instruction. For example, if the image data enhancement operation indicated by the instruction preceding the image data enhancement instruction is of the type involving size change (e.g., resizing, cropping), then the image description information of the image to be processed corresponding to the image data enhancement instruction will be changed accordingly relative to the image description information corresponding to the decrypted image. If the image data enhancement operation indicated by the instruction preceding the image data enhancement instruction is not of the type involving size change (e.g., flipping, blurring), then the image description information of the image to be processed corresponding to the image data enhancement instruction can be consistent with the image description information corresponding to the decrypted image.

[0046] In step 222, data enhancement processing is performed on the corresponding images to be processed according to the obtained image data enhancement instructions to obtain the processed image set.

[0047] In this embodiment, a fully filled image data enhancement instruction typically includes the type of image data enhancement operation indicated by the instruction, the image to be processed targeted by the instruction, and the size information of the image to be processed targeted by the instruction. Thus, data enhancement processing can be performed on the image to be processed corresponding to the obtained image data enhancement instruction according to the type of image data enhancement operation indicated by each image data enhancement instruction, resulting in a processed image set.

[0048] Optionally, refer to Figure 3 , Figure 3 A schematic diagram of an example of a chip 300 for performing an image data enhancement processing procedure according to an embodiment of this specification is shown.

[0049] like Figure 3 As shown, the chip described above may include an instruction parser 310, a data augmentation operator module 320, a multiplexer 330, and an image data buffer 340. In one example, the data augmentation operator module 320 may include various types of operator modules, such as a crop operator module 321, a flip operator module 322, a normalize operator module 323, and a resize operator module 324. In one example, the aforementioned operator module may include a hardware module developed using Verilog. The instruction parser 310 may send control information to the data augmentation operator module 320 and the multiplexer 330, respectively. The multiplexer 330 may open or close the data transmission channel connected to the data augmentation operator module 320 or the image data buffer 340 based on the control information received from the instruction parser 310. The specific operation of the instruction parser 310, the data augmentation operator module 320, the multiplexer 330, and the image data buffer 340 can be referred to the following description.

[0050] In one example, the chip described above could be an ASIC chip or an FPGA chip. It can be understood that, in one example, the chip environment described above can be considered a secure computing environment. This allows for the provision of a hardware-based image data enhancement solution suitable for a secure computing environment.

[0051] Continue to refer to Figure 4 , Figure 4 A flowchart illustrating an example of a data augmentation process 400 according to an embodiment of this specification is shown.

[0052] At 410, for each image data enhancement instruction, the obtained image data enhancement instruction is parsed by the instruction parser to obtain the parsing result.

[0053] In this embodiment, the acquired image data enhancement instructions can be parsed by an instruction parser to obtain the parsing results for each image data enhancement instruction. The parsing results may include startup control information and channel selection control information obtained from the parsed image data enhancement instructions. Specifically, the startup control information can be used to instruct the activation of a data enhancement operator matching the type of image data enhancement operation of the image data enhancement instruction. The channel selection control information can be used to instruct the opening of a data transmission channel connected to a data enhancement operator matching the type of image data enhancement operation of the image data enhancement instruction.

[0054] At 420, the instruction parser sends the start control information and channel selection control information corresponding to the image data enhancement instruction to the data enhancement operator module and the multiplexer, respectively.

[0055] At 430, for each image data enhancement instruction, a data transmission channel connected to a data enhancement operator module that matches the type of the image data enhancement operation is opened via a multiplexer according to the channel selection control information.

[0056] In one example, if the image data enhancement operation type of the image data enhancement instruction is flip, a data transmission channel connected to the flip operator module can be opened via a multiplexer according to the channel selection control information. In another example, if the image data enhancement operation type of the image data enhancement instruction is crop, a data transmission channel connected to the crop operator module can be opened via a multiplexer according to the channel selection control information. In yet another example, the multiplexer can continuously open the data transmission channel connected to the image data buffer to transmit the corresponding image to be processed to the corresponding data enhancement operator module and to transmit the processed image received from the data enhancement operator module. In yet another example, the image data buffer can be used to store the image to be processed and / or the processed image. The image to be processed can be obtained by decrypting an encrypted image from an insecure computing environment.

[0057] Optionally, the aforementioned image data buffer may include a source image data buffer and a result image data buffer. For each image data enhancement instruction, a data transmission channel connected to the source image data buffer or the result image data buffer can be opened via a multiplexer according to the channel selection control information. In one example, the source image data buffer may be used to store the original image to be processed, i.e., the image to be processed targeted by the first image data enhancement instruction in the complete image data enhancement instruction sequence. For example, the original image to be processed may be obtained by a higher-level subsystem (e.g., an image decryption device) after decrypting an encrypted image from an insecure computing environment. The result image data buffer may be used to store the image after image processing enhancement. In one example, the channel selection control information may also be used to instruct the opening of a data transmission channel connected to the source image data buffer or the result image data buffer. In one example, the result image data buffer may also provide an interface for reading data, allowing a lower-level subsystem (e.g., a deep learning module) to read the processed image data.

[0058] At 440, for each image data enhancement instruction, a data enhancement operator module that matches the type of the image data enhancement operation responds to the start control information to perform data enhancement processing on the image to be processed corresponding to the image data enhancement instruction to be filled, obtained from the image data buffer, and generate a processed image set.

[0059] In this embodiment, for each image data enhancement instruction, the data transmission channel opened by the multiplexer can transmit the image to be processed corresponding to the image data enhancement instruction to the data enhancement operator module that matches the type of image data enhancement operation. Then, the data enhancement operator module matching the type of image data enhancement operation responds to the start control information and performs data enhancement processing on the corresponding image to be processed, generating a processed image. Optionally, the processed image can also be transmitted to an image data buffer for storage through the data transmission channel opened by the multiplexer.

[0060] Optionally, at 450, the processed images are stored in the result image data buffer via the data transmission channel opened by the multiplexer. This allows the multiplexer to store the source image data and the processed image data separately, improving the query efficiency between different data buffers.

[0061] The following is for reference. Figure 5 , Figure 5 A schematic diagram of yet another example of a chip 500 for performing an image data enhancement process according to an embodiment of this specification is shown.

[0062] like Figure 5As shown, the chip described above may include an instruction parser 510, a data enhancement operator module 520, a multiplexer 530, a source image data buffer 540, an intermediate result image data buffer 550, and a final result image data buffer 560. In one example, the multiplexer 530 can open or close the data transmission channel connected to the source image data buffer 540, the intermediate result image data buffer 550, or the final result image data buffer 560 based on the control information received from the instruction parser 510. The instruction parser 510, the data enhancement operator module 520, and the multiplexer 530 can be referenced from the foregoing. Figure 3 The descriptions of the instruction parser 310, data augmentation operator module 320, and multiplexer 330 in the embodiments will not be repeated here.

[0063] In this embodiment, the source image data buffer 540 can be used to store the original image to be processed, that is, the image to be processed corresponding to the first image data enhancement instruction in the fully filled image data enhancement instruction sequence. The intermediate result image data buffer 550 can be used to store intermediate results, that is, the processed image obtained by instructions other than the last instruction in the fully filled image data enhancement instruction sequence. The final result image data buffer 560 can be used to store the final result, that is, the processed image obtained by the last instruction in the fully filled image data enhancement instruction sequence. In one example, if the fully filled data enhancement instruction sequence includes two or more instructions, the image to be processed corresponding to the last instruction in the data enhancement instruction sequence can be stored in the intermediate result image data buffer 550. In one example, if the fully filled data enhancement instruction sequence includes three or more instructions, the image to be processed corresponding to instructions other than the last instruction in the data enhancement instruction sequence can be stored in the intermediate result image data buffer 550.

[0064] Continue to refer to Figure 6 , Figure 6 A schematic diagram of an example of an image data augmentation instruction 600 to be filled, according to an embodiment of this specification, is shown.

[0065] like Figure 6As shown, the image data augmentation instruction to be filled may include the type of image data augmentation operation, the field to be filled, and source image data buffer indication information indicating whether the image to be processed is located in the source image data buffer or the intermediate result image data buffer. The field to be filled may include a field indicating the size of the image to be processed. In one example, the bit width of the image data augmentation instruction to be filled may be 256 bits. In one example, the value of the type of image data augmentation operation may include, for example, "1", "2", "3", etc. "1" can be used to indicate a flip operation. "2" can be used to indicate a cropping operation. "3" can be used to indicate a normalization operation. In one example, the source image data buffer indication information may include, for example, "A" and "B". "A" can be used to indicate that the image to be processed is located in the source image data buffer. "B" can be used to indicate that the image to be processed is located in the intermediate result image data buffer. In one example, the value of the field to be filled may be empty or a preset placeholder may be used.

[0066] In one example, the other bits can have different meanings depending on the type of image data augmentation operation. For example, for a cropping operation instruction, the four sets of 12 bits in the diagram can be used to represent the size of the cropping box (e.g., width and height) and the position of the cropping box (e.g., the horizontal and vertical coordinates of the center point of the cropping box).

[0067] In one example, a set of 32 bits in the image data augmentation instruction can be used to represent the starting address of the storage location of the image to be processed corresponding to the instruction.

[0068] In one example, the image data augmentation instruction to be filled may also include reserved bits. This can be flexibly configured according to actual needs.

[0069] Continue to refer to Figure 7 , Figure 7 A schematic diagram of an example of a fill-in image data enhancement instruction 700 according to an embodiment of this specification is shown.

[0070] like Figure 7As shown, the aforementioned image data augmentation instruction for filling can include the already filled fields in the image data augmentation instruction to be filled, and the width and height of the image to be processed filled according to the image description information of the image to be processed corresponding to the instruction. In one example, the image data augmentation instruction for filling can also include result image data buffer indication information for indicating whether the processed image is located in the intermediate result image data buffer or the final result image data buffer. In one example, another set of 32 bits in the image data augmentation instruction to be filled can be used to represent the starting address for storing the image processed according to the instruction. Thus, in privacy-preserving scenarios, the instruction can be filled completely according to the size information of the image to be processed obtained in a secure computing environment.

[0071] In this embodiment, in one example, the value of the result image data buffer indication information may include, for example, "B" or "C". "B" can be used to indicate that the image processed according to the instruction will be stored in the intermediate result image data buffer. "C" can be used to indicate that the image processed according to the instruction will be stored in the final result image data buffer.

[0072] The following is for reference. Figure 8 , Figure 8 A schematic diagram of yet another example of a chip 800 for performing an image data enhancement method according to an embodiment of this specification is shown.

[0073] like Figure 8 As shown, the chip described above may include an instruction parser 810, a data augmentation operator module 820, a multiplexer 830, a source image data buffer 840, an intermediate result image data buffer 850, a final result image data buffer 860, and a result description information buffer 870. The instruction parser 810, data augmentation operator module 820, multiplexer 830, source image data buffer 840, intermediate result image data buffer 850, and final result image data buffer 860 can be referenced from the aforementioned... Figure 5 The descriptions of the instruction parser 510, data augmentation operator module 520, multiplexer 530, source image data buffer 540, intermediate result image data buffer 550, and final result image data buffer 560 in the embodiment will not be repeated here.

[0074] In this embodiment, the aforementioned result image data buffer indication information can be used to indicate that the processed image is located in the final result image data buffer. The aforementioned parsing result may also include result description information of the processed image. The aforementioned result description information may include the storage address of the processed image and the size information of the processed image. In one example, the processed image can be stored in the final result image data buffer 860 via the data transmission channel opened by the multiplexer 830. The aforementioned result description information can also be stored in the result description information buffer 870 via the instruction parser 810.

[0075] Optionally, the chip may further include a FIFO buffer 880 for pending image data augmentation instructions, a FIFO buffer 890 for pending image description information, a FIFO buffer 8100 for complete instructions, and an instruction preprocessor 8110. The FIFO buffer 880 for pending image data augmentation instructions can be used to cache various pending image data augmentation instructions. In one example, the FIFO buffer 880 can be used to cache a sequence of pending image data augmentation instructions downloaded from a host computer. The FIFO buffer 890 for pending image description information can be used to send the image description information of the pending image to the instruction preprocessor 8110. The FIFO buffer 8100 for complete instructions can be used to send the image data augmentation instructions to the instruction parser 810. In one example, the specific operation of the instruction preprocessor 8110 can be referred to the foregoing. Figure 2 The relevant description of step 221 will not be repeated here. This provides a complete solution for hardware-based instruction padding and image data enhancement.

[0076] Continue to refer to Figure 9 , Figure 9 A schematic diagram of an example of a system-on-chip 900 for performing a method processing procedure for image data enhancement according to an embodiment of this specification is shown.

[0077] like Figure 9 As shown, the system-on-a-chip 900 may include an FPGA chip 910 and a processor hard core 920 connected to the FPGA chip 910. The FPGA chip 910 may include an instruction parser 911, a data augmentation operator module 912, a multiplexer 913, and an image data buffer 914. In one example, the FPGA chip 910 may refer to the aforementioned... Figures 3-8 The chip described in the example.

[0078] In this embodiment, a target image data enhancement instruction that meets the operator unloading conditions can be obtained via the instruction parser 911. In one example, the operator unloading condition may be an operator of a predetermined type (e.g., with relatively complex computational logic). In another example, the operator unloading condition may be a tight processing time limit. Thus, the target image data enhancement instruction is a fully filled image data enhancement instruction that meets the operator unloading conditions. Then, the instruction parser 911 can send an image processing request to the processor hard core according to the target image data enhancement instruction, so that the processor hard core processes the image to be processed according to the image data enhancement processing method indicated by the target image data enhancement instruction. Therefore, when facing more complex enhancement operations or tight deadlines, some operators can be unloaded onto the processor hard core 920 connected to the FPGA chip 910 for processing, thereby improving the overall processing efficiency of the image data enhancement operation by utilizing the processor hard core 920. Afterwards, the FPGA chip 910 can receive the image processing result corresponding to the image processing request returned by the processor hard core in various ways.

[0079] In one example, the image processing result may include the processed image corresponding to the image processing request. In another example, the image processing result may also include image description information of the processed image corresponding to the image processing request. Therefore, the processed image corresponding to the image processing request can be stored in the image data buffer 914. Optionally, the FPGA chip may also include a result description information buffer. Therefore, the image description information of the processed image corresponding to the image processing request can also be stored in the result description information buffer.

[0080] use Figures 1-9 The method for image data augmentation disclosed herein can achieve the effect of image data augmentation in a secure computing environment by filling a sequence of image data augmentation instructions from an insecure computing environment in a secure computing environment.

[0081] The following is for reference. Figure 10 , Figure 10 A block diagram illustrating an example of an apparatus 1000 for image data enhancement according to an embodiment of this specification is shown. This apparatus embodiment can be used with... Figures 2-9 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0082] like Figure 10 As shown, the apparatus 1000 for image data enhancement may include a fill instruction acquisition unit 1010, an instruction filling unit 1020, and a data enhancement unit 1030.

[0083] The image data augmentation instruction acquisition unit 1010 is configured to acquire a sequence of image data augmentation instructions to be augmented from an insecure computing environment. Each image data augmentation instruction includes a type of image data augmentation operation and a field to be augmented, wherein the field to be augmented includes a field indicating the size of the image to be processed. The operation of the image data augmentation instruction acquisition unit 1010 can be referred to above. Figure 2 The operation described in section 210.

[0084] The instruction filling unit 1020 is configured, in a secure computing environment, to fill in the fields to be filled for each image data augmentation instruction to be filled, based on the obtained image description information of the image to be processed corresponding to the instruction, to obtain a fully filled image data augmentation instruction. The image description information of the image to be processed is obtained by decrypting an encrypted image from an insecure computing environment, and includes size information. The operation of the instruction filling unit 1020 can be referred to above. Figure 2 The operation described in section 221.

[0085] The data augmentation unit 1030 is configured to perform data augmentation processing on the corresponding images to be processed according to the obtained image data augmentation instructions in a secure computing environment, thereby obtaining a processed image set. The operation of the data augmentation unit 1030 can be referred to above. Figure 2 The operation described in section 222.

[0086] Further reference Figure 11 , Figure 11 A block diagram of yet another example of an apparatus 1100 for image data enhancement according to an embodiment of this specification is shown.

[0087] In one example, the image data enhancement apparatus 1100 may include a fill instruction acquisition unit 1110, an instruction filling unit 1120, and a chip. The chip may include a data enhancement unit 1130 and an image data buffer 1140. The data enhancement unit 1130 includes an instruction parser 1131, a data enhancement operator module 1132, and a multiplexer 1133. The instruction parser 1131 is configured to parse the acquired image data enhancement instruction for each image data enhancement instruction to obtain a parsing result; and send start control information and channel selection control information corresponding to the image data enhancement instruction to the data enhancement operator module and the multiplexer, respectively, wherein the parsing result includes the start control information and the channel selection control information. The multiplexer 1133 is configured to, for each image data enhancement instruction, open a data transmission channel connected to the data enhancement operator module matching the type of the image data enhancement operation according to the channel selection control information. The data augmentation operator module 1132, which matches the type of image data augmentation operation, is configured to perform data augmentation processing on the image to be processed corresponding to the image data augmentation instruction to be filled, obtained from the image data buffer 1140, in response to the start control information, for each image data augmentation instruction, thereby generating a processed image set. Figure 10 The corresponding descriptions of the instruction acquisition unit 1010 and instruction filling unit 1020 in the embodiments will not be repeated here.

[0088] In one example, the image data buffer includes a source image data buffer and a result image data buffer. The multiplexer can also be configured to, for each image data enhancement instruction, open a data transmission channel connected to either the source image data buffer or the result image data buffer based on channel selection control information; and store each processed image into the result image data buffer via the opened data transmission channel. The operation of the multiplexer described above can be referred to the foregoing. Figure 4 The embodiments include optional implementations of step 430 and a description of step 450.

[0089] In one example, the result image data buffer includes an intermediate result image data buffer and a final result image data buffer. The result image data buffer indication information is used to indicate that the processed image is located in the final result image data buffer. The chip also includes a result description information buffer. The parsing result also includes result description information of the processed image, which includes the storage address of the processed image and the size information of the processed image. The multiplexer is further configured to store the processed image into the final result image data buffer via an enabled data transmission channel. The instruction parser is further configured to store the result description information into the result description information buffer. The operation of the multiplexer, result description information buffer, and instruction parser described above can be referred to the foregoing. Figure 8 The corresponding descriptions in the embodiments.

[0090] In one example, the chip further includes a FIFO buffer for instructions to be filled, a FIFO buffer for image description information to be processed, a FIFO buffer for complete instructions, and an instruction preprocessor. The FIFO buffer for instructions to be filled is used to buffer each image data enhancement instruction to be filled. The FIFO buffer for image description information to be processed is used to send the image description information of the image to be processed to the instruction preprocessor. The FIFO buffer for complete instructions is used to send the image data enhancement instructions to the instruction parser. The instruction preprocessor can be consistent with the description of the instruction filling unit. The operation of the FIFO buffer for instructions to be filled, the FIFO buffer for image description information to be processed, the FIFO buffer for complete instructions, and the instruction preprocessor can be referred to the foregoing. Figure 8 The corresponding descriptions of the optional implementation methods in the embodiments.

[0091] Continue to refer to Figure 12 , Figure 12 A block diagram of yet another example of an apparatus 1200 for image data enhancement according to an embodiment of this specification is shown.

[0092] In one example, the image data enhancement device 1200 may include a system-on-a-chip (SoC) encapsulating an FPGA chip 1201 and a processor hard core 1202 connected to the FPGA chip 1201. The FPGA chip 1201 may include a fill instruction fetching unit 1210, an instruction filling unit 1220, a data enhancement unit 1230, and an image data buffer 1240. The data enhancement unit 1230 may include an instruction parser 1231, a data enhancement operator module 1232, and a multiplexer 1233. The specific operation of the instruction parser 1231, the data enhancement operator module 1232, and the multiplexer 1233 can be found in the foregoing. Figure 11The descriptions of the instruction parser 1131, data augmentation operator module 1132, and multiplexer 1133 in the embodiments are not repeated here. The instruction parser 1231 is further configured to acquire a target image data augmentation instruction that satisfies the operator unloading condition; and to send an image processing request to the processor hard core 1202 according to the target image data augmentation instruction, so that the processor hard core 1202 processes the image to be processed according to the image data augmentation processing method indicated by the target image data augmentation instruction. The device 1200 for image data augmentation may further include a result receiving unit 1203, configured to receive the image processing result returned by the processor hard core 1202 corresponding to the image processing request.

[0093] In one example, the result receiving unit 1203 described above may be a separate module located within the image data enhancement device 1200, independent of the FPGA chip 1201 and the processor hard core 1202, or it may be a module integrated into the FPGA chip 1201. For example, when the multiplexer 1233 or the image data buffer 1240 can be further configured to receive the image processing result returned by the processor hard core corresponding to the image processing request, then the multiplexer 1233 or the image data buffer 1240 can serve as the result receiving unit 1203 described above.

[0094] Continue to refer to Figure 13 , Figure 13 A block diagram of an example image processing system 1300 according to an embodiment of this specification is shown.

[0095] like Figure 13 As shown, the image processing system 1300 may include: an image decryption device 1310, an image data enhancement device 1320, a deep learning device 1330, and a communication device 1340. The image decryption device 1310 is configured to decrypt encrypted images from an insecure computing environment to obtain the image to be processed and the corresponding image description information. In one example, the image to be processed and the corresponding image description information can be referred to the foregoing. Figure 2 The corresponding descriptions in steps 210 and 221 of the embodiments are provided. The apparatus 1320 for image data enhancement can be referred to the foregoing. Figures 10-12The following is a description of the embodiments. A deep learning device 1330 is configured to perform calculations on the processed image using deep learning operator modules to obtain image calculation results. In one example, the deep learning operator modules may include, for example, a convolution module, an activation module, etc. A communication device 1340 is used to provide a communication channel between the image processing system 1300 and other electronic devices. In one example, the image processing system may be an FPGA chip, and the communication device 1340 may be used to provide a communication channel between the FPGA chip and a host computer.

[0096] Further reference Figure 14 , Figure 14 A block diagram of an example image decryption apparatus 1400 in an image processing system according to an embodiment of this specification is shown.

[0097] like Figure 14 As shown, the image decryption device 1400 may include an image decryption unit 1410 and an image decoding unit 1420. The image decryption unit 1410 is configured to decrypt an encrypted image from an insecure computing environment to obtain plaintext compressed data of the image. The image decoding unit 1420 is configured to decode the plaintext compressed data of the image to obtain the image to be processed and corresponding image description information.

[0098] In one example, the image decryption unit 1410 can decrypt the ciphertext image entering the secure computing environment to obtain the plaintext compressed data of the image. For example, the ciphertext image (i.e., the encrypted image) can be XORed with the key image to achieve the decryption process, obtaining the plaintext compressed data of the image (e.g., a JPEG format compressed data packet). Furthermore, the image decoding unit 1420 can perform image decoding on the plaintext compressed data of the image according to the compression method of different image types to obtain the image to be processed and the corresponding image description information.

[0099] use Figures 13-14 The image processing system disclosed herein can use the decrypted image obtained by the image decryption device, and then perform data enhancement through the image data enhancement device to enrich the sample data. In turn, the processed image set after image data enhancement can be provided to the deep learning device composed of typical deep learning operators, thereby providing a deep learning training or inference framework that can be used in privacy protection scenarios.

[0100] Reference above Figures 1 to 14 Embodiments of methods, apparatus, and systems for image data enhancement according to examples of this specification have been described.

[0101] The image data enhancement apparatus described in this specification can be implemented in hardware, software, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of its host device reading the corresponding computer program instructions from the memory into memory and executing them. In the embodiments of this specification, the image data enhancement apparatus can, for example, be implemented using an electronic device.

[0102] Figure 15 A schematic diagram of an apparatus 1500 for image data enhancement according to an embodiment of this specification is shown.

[0103] like Figure 15 As shown, the apparatus 1500 for image data enhancement may include at least one processor 1510, a memory (e.g., non-volatile memory) 1520, a memory 1530, and a communication interface 1540, and the at least one processor 1510, memory 1520, memory 1530, and communication interface 1540 are connected together via a bus 1550. At least one processor 1510 executes at least one computer-readable instruction (i.e., the elements implemented in software described above) stored or encoded in the memory.

[0104] In one embodiment, computer-executable instructions are stored in memory that, when executed, cause at least one processor 1510 to: acquire a sequence of image data augmentation instructions to be filled from an insecure computing environment, wherein each image data augmentation instruction includes a type of image data augmentation operation and a field to be filled, the field to be filled including a field indicating the size of the image to be processed; and perform the following image data augmentation operations in a secure computing environment: for each image data augmentation instruction to be filled, fill the field to be filled of the image data augmentation instruction according to the acquired image description information of the image to be processed corresponding to the image data augmentation instruction to be filled, to obtain a fully filled image data augmentation instruction, wherein the image description information of the image to be processed is obtained based on the result of decrypting an encrypted image from the insecure computing environment, the image description information including size information; and perform data augmentation processing on the corresponding image to be processed according to the obtained image data augmentation instructions to obtain a processed image.

[0105] It should be understood that the computer-executable instructions stored in memory, when executed, cause at least one processor 1510 to perform the above-described combinations in the various embodiments of this specification. Figure 1-9 The description includes various operations and functions.

[0106] According to one embodiment, a program product, such as a computer-readable medium, is provided. The computer-readable medium may have instructions (i.e., the elements implemented in software as described above), which, when executed by a computer, cause the computer to perform the above-described combinations of the various embodiments of this specification. Figure 1-9 The description includes various operations and functions.

[0107] Specifically, a system or apparatus equipped with a readable storage medium may be provided, on which software program code implementing the functions of any of the embodiments described above is stored, and the computer or processor of the system or apparatus can read and execute the instructions stored in the readable storage medium.

[0108] In this case, the program code itself, which can be read from a readable medium, can perform the functions of any of the above embodiments. Therefore, the machine-readable code and the readable storage medium storing the machine-readable code constitute a part of the present invention.

[0109] The computer program code required for the operation of each part of this manual can be written in any one or more programming languages, including object-oriented programming languages ​​such as Java, Scala, Smalltalk, Eiffel, JADE, Emerald, C++, C#, VB, .NET, and Python; conventional procedural programming languages ​​such as C, Visual Basic 2003, Perl, COBOL 2002, PHP, and ABAP; dynamic programming languages ​​such as Python, Ruby, and Groovy; or other programming languages. This program code can run on the user's computer, or as a standalone software package on the user's computer, or partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter case, the remote computer can be connected to the user's computer via any network, such as a local area network (LAN) or wide area network (WAN), or connected to an external computer (e.g., via the Internet), or in a cloud computing environment, or used as a service, such as Software as a Service (SaaS).

[0110] Examples of readable storage media include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD-RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer or the cloud via a communication network.

[0111] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0112] Not all steps and units in the above process and system structure diagrams are mandatory; some steps or units can be omitted as needed. The execution order of each step is not fixed and can be determined as required. The device structure described in the above embodiments can be a physical structure or a logical structure. That is, some units may be implemented by the same physical entity, or some units may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0113] The term "exemplary" as used throughout this specification means "serving as an example, instance, or illustration" and does not imply that it is "preferred" or "advantageous" over other embodiments. Detailed descriptions are included for the purpose of providing an understanding of the described techniques. However, these techniques may be practiced without these detailed descriptions. In some instances, well-known structures and apparatuses are shown in block diagram form to avoid obscuring the concepts of the described embodiments.

[0114] The optional embodiments of the present specification have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present specification are not limited to the specific details in the above embodiments. Within the scope of the technical concept of the embodiments of the present specification, various simple modifications can be made to the technical solutions of the embodiments of the present specification, and these simple modifications all fall within the protection scope of the embodiments of the present specification.

[0115] The foregoing description of this specification is provided to enable any person skilled in the art to implement or use the content of this specification. Various modifications to the content of this specification will be apparent to those skilled in the art, and the general principles defined herein can be applied to other variations without departing from the scope of protection of this specification. Therefore, this specification is not limited to the examples and designs described herein, but is consistent with the widest scope of the principles and novel features disclosed herein.

Claims

1. A method for image data augmentation, comprising: Obtain a sequence of image data augmentation instructions to be filled from an insecure computing environment, wherein each image data augmentation instruction includes a type of image data augmentation operation and a field to be filled, the field to be filled including a field for indicating the size of the image to be processed; Perform the following image data augmentation operations in a secure computing environment: For each image data augmentation instruction to be filled, the field to be filled of the image data augmentation instruction is filled according to the obtained image description information of the image to be processed corresponding to the image data augmentation instruction to be filled, so as to obtain a fully filled image data augmentation instruction. The image description information of the image to be processed is obtained according to the result of decrypting the ciphertext image from the insecure computing environment, and the image description information includes size information. Based on the obtained image data enhancement instructions, the corresponding images to be processed are subjected to data enhancement processing to obtain the processed image set.

2. The method as described in claim 1, wherein, The process involves performing data enhancement processing on the corresponding images to be processed according to the obtained image data enhancement instructions. The resulting processed image is then executed by the chip, which includes an instruction parser, a data enhancement operator module, a multiplexer, and an image data buffer. The process of performing data enhancement processing on the corresponding images to be processed according to the obtained image data enhancement instructions to obtain the processed image set includes: For each image data enhancement instruction, the instruction parser parses the acquired image data enhancement instruction to obtain a parsing result; and sends the start control information and channel selection control information corresponding to the image data enhancement instruction to the data enhancement operator module and the multiplexer, respectively, wherein the parsing result includes the start control information and the channel selection control information; For each image data enhancement command, the data transmission channel connected to the data enhancement operator module matching the type of the image data enhancement operation is opened via the multiplexer according to the channel selection control information; and For each image data enhancement instruction, a data enhancement operator module that matches the type of the image data enhancement operation responds to the start control information to perform data enhancement processing on the image to be processed corresponding to the image data enhancement instruction to be filled, which is obtained from the image data buffer, and generates a processed image set.

3. The method as described in claim 2, wherein, The image data cache includes a source image data cache and a result image data cache. The step of performing data enhancement processing on the corresponding images to be processed according to the obtained image data enhancement instructions to obtain the processed image set further includes: For each image data enhancement instruction, the data transmission channel connected to the source image data buffer or the result image data buffer is opened via the multiplexer according to the channel selection control information. Each processed image is stored in the result image data buffer via the data transmission channel opened by the multiplexer.

4. The method of claim 3, wherein, The result image data cache includes an intermediate result image data cache and a final result image data cache. The image data enhancement instruction to be filled also includes source image data cache indication information for indicating that the image to be processed is located in the source image data cache or the intermediate result image data cache. The image data enhancement instruction also includes result image data cache indication information for indicating that the processed image is located in the intermediate result image data cache or the final result image data cache.

5. The method of claim 4, wherein, The result image data buffer indication information is used to indicate that the processed image is located in the final result image data buffer. The chip also includes a result description information buffer. The parsing result also includes result description information of the processed image, which includes the storage address and size information of the processed image. The process of storing the processed image into the result image data buffer via the data transmission channel opened by the multiplexer includes: The processed image is stored in the final result image data buffer via the data transmission channel opened by the multiplexer. The image data enhancement operation also includes: The result description information is stored in the result description information cache via the instruction parser.

6. The method of claim 5, wherein, The chip also includes a FIFO buffer for instructions to be filled, a FIFO buffer for image description information to be processed, a FIFO buffer for complete instructions, and an instruction preprocessor. The FIFO buffer for instructions to be filled is used to cache each image data enhancement instruction to be filled. The FIFO buffer for image description information to be processed is used to send the image description information of the image to be processed to the instruction preprocessor. The FIFO buffer for complete instructions is used to send the image data enhancement instructions to the instruction parser.

7. The method as described in any one of claims 2 to 6, wherein, The chip includes a system-on-a-chip that encapsulates an FPGA chip and a processor hard core connected to the FPGA chip. The FPGA chip includes an instruction parser, a data augmentation operator module, a multiplexer, and an image data buffer. The image data enhancement operation also includes: The target image data enhancement instruction that meets the operator unloading condition is obtained through the instruction parser; an image processing request is sent to the processor hard core according to the target image data enhancement instruction, so that the processor hard core processes the image to be processed in accordance with the image data enhancement processing method indicated by the target image data enhancement instruction; Receive the image processing result returned by the processor hard core corresponding to the image processing request.

8. An apparatus for image data enhancement, comprising: The image data augmentation instruction acquisition unit is configured to acquire a sequence of image data augmentation instructions to be filled from an insecure computing environment, wherein each image data augmentation instruction to be filled includes a type of image data augmentation operation and a field to be filled, the field to be filled including a field for indicating the size of the image to be processed; The instruction filling unit is configured to, in a secure computing environment, fill the fields to be filled in each image data augmentation instruction to be filled in according to the obtained image description information of the image to be processed corresponding to the image data augmentation instruction, so as to obtain a fully filled image data augmentation instruction. The image description information of the image to be processed is obtained according to the result of decrypting the ciphertext image from the insecure computing environment, and the image description information includes size information. The data augmentation unit is configured to perform data augmentation processing on the corresponding images to be processed according to the obtained image data augmentation instructions in a secure computing environment, so as to obtain a processed image set.

9. The apparatus of claim 8, wherein, The device includes a chip, which includes the data enhancement unit and an image data buffer. The data enhancement unit includes an instruction parser, a data enhancement operator module, and a multiplexer. The instruction parser is configured to parse the acquired image data enhancement instruction for each image data enhancement instruction to obtain the parsing result; The start control information and channel selection control information corresponding to the image data enhancement instruction are sent to the data enhancement operator module and the multiplexer, respectively, wherein the parsing result includes the start control information and the channel selection control information; The multiplexer is configured to, for each image data enhancement instruction, open a data transmission channel connected to a data enhancement operator module matching the type of the image data enhancement operation, based on the channel selection control information; and A data augmentation operator module that matches the type of image data augmentation operation is configured to perform data augmentation processing on the image to be processed corresponding to the image data augmentation instruction to be filled, obtained from the image data buffer, in response to the start control information for each image data augmentation instruction, and generate a processed image set.

10. The apparatus of claim 9, wherein, The image data cache includes a source image data cache and a result image data cache. The multiplexer is also configured to, for each image data enhancement instruction, open a data transmission channel connected to the source image data buffer or the result image data buffer according to the channel selection control information; and store each processed image into the result image data buffer via the opened data transmission channel.

11. The apparatus of claim 10, wherein, The result image data cache includes an intermediate result image data cache and a final result image data cache. The result image data cache indicator information indicates that the processed image is located in the final result image data cache. The chip also includes a result description information cache. The parsing result further includes result description information of the processed image, which includes the storage address and size information of the processed image. The multiplexer is also configured to store the processed image into the final result image data buffer via the opened data transmission channel; The instruction parser is also configured to store the result description information in the result description information cache.

12. The apparatus of claim 11, wherein, The chip also includes a FIFO buffer for instructions to be filled, a FIFO buffer for image description information to be processed, a FIFO buffer for complete instructions, and an instruction preprocessor. The FIFO buffer for instructions to be filled is used to cache each image data enhancement instruction to be filled. The FIFO buffer for image description information to be processed is used to send the image description information of the image to be processed to the instruction preprocessor. The FIFO buffer for complete instructions is used to send the image data enhancement instructions to the instruction parser.

13. The apparatus according to any one of claims 9 to 12, wherein, The device includes a system-on-a-chip (SoC) that encapsulates an FPGA chip and a processor hard core connected to the FPGA chip. The FPGA chip includes a command fetching unit, a command filling unit, a data enhancement unit, and an image data buffer. The instruction parser is further configured to obtain a target image data enhancement instruction that satisfies the operator unloading condition; and to send an image processing request to the processor hard core according to the target image data enhancement instruction, so that the processor hard core processes the image to be processed in accordance with the image data enhancement processing method indicated by the target image data enhancement instruction. The device further includes: The result receiving unit is configured to receive the image processing result corresponding to the image processing request returned by the processor hard core.

14. An image processing system, comprising: The image decryption device is configured to decrypt ciphertext images from an insecure computing environment to obtain the image to be processed and the corresponding image description information. The apparatus for image data enhancement as described in any one of claims 8 to 13; A deep learning device is configured to perform calculations on the processed image using deep learning operator modules to obtain image calculation results; and Communication device.

15. The image processing system of claim 14, wherein, The image decryption device includes: The image decryption unit is configured to decrypt ciphertext images from an insecure computing environment to obtain plaintext compressed data of the image. The image decoding unit is configured to perform image decoding on the plaintext compressed data of the image to obtain the image to be processed and the corresponding image description information.

16. A computer-readable storage medium storing a computer program that, when executed, causes a processor to perform the method for image data enhancement as described in any one of claims 1 to 7.

17. A computer program product comprising a computer program that, when executed, causes a processor to perform the method for image data enhancement as described in any one of claims 1 to 7.

Images