Data processing system and method

Abstract

A data processing system comprising a storage device is disclosed. The data processing system further comprises at least one processor to generate output data using at least a portion of a first neural network layer and to generate a key associated with the at least a portion of the first neural network layer. The at least one processor is further operable to obtain the key from the storage device and to obtain a version of the output data for input into the second neural network layer. Using the key, the at least one processor is further operable to determine whether the version of the output data is different from the output data.

Description

Background Technology Technical Field

[0001] This disclosure relates to a data processing system and method.

[0002] Related technical specifications

[0003] Neural network computations can be performed in a data processing system that includes a microprocessor such as a neural processing unit (NPU), a central processing unit (CPU), and a graphics processing unit (GPU). The aim is to improve the security of neural network computations. Summary of the Invention

[0004] According to a first aspect of this disclosure, a data processing system is provided, comprising: a storage device; and at least one processor, the at least one processor being configured to: generate output data using at least a portion of a first neural network layer; generate a key associated with the at least a portion of the first neural network layer; obtain the key from the storage device; obtain a version of the output data for input into a second neural network layer; and use the key to determine whether the version of the output data is different from the output data.

[0005] According to a second aspect of this disclosure, a method is provided, the method comprising: generating output data using at least a portion of a first neural network layer; generating a key associated with the at least a portion of the first neural network layer; storing the key in a storage device; obtaining the key from the storage device; obtaining a version of the output data for input into a second neural network layer; and using the key to determine whether the version of the output data is different from the output data. Attached Figure Description

[0006] Further features and advantages will become apparent from the following description, given by way of example only with reference to the accompanying drawings.

[0007] Figure 1 This is a schematic diagram illustrating a method for determining, according to an example, whether the output data generated by the first neural network layer is different from the version of the output data used as input to the second neural network layer;

[0008] Figure 2 This is a schematic diagram illustrating a method for determining, according to another example, whether the output data generated by the first neural network layer is different from the version of the output data used as input to the second neural network layer;

[0009] Figure 3 This is a schematic diagram illustrating a method for determining, according to another example, whether the output data generated by the first neural network layer is different from the version of the output data used as input to the second neural network layer;

[0010] Figure 4 This is a schematic diagram of the image processing system based on the example;

[0011] Figure 5 It includes Figure 4 A schematic diagram of the internal components of an exemplary computing system for an image processing system;

[0012] Figure 6 This is a flowchart illustrating a method for sending a key to a data processing system according to an example;

[0013] Figure 7 It includes Figure 4 A schematic diagram of the internal components of another exemplary computing system for an image processing system;

[0014] Figure 8 This is a flowchart illustrating a method for determining, based on an example, whether the input data used for input to the first neural network layer is different from the version of the input data used for input to the first neural network layer;

[0015] Figure 9 It is a schematic diagram of a system including a network, based on the example; and

[0016] Figure 10 This is a schematic diagram of an image to be processed using an image processing system, based on an example. Detailed Implementation

[0017] Referring to the accompanying drawings, the details of the systems and methods according to the examples will become apparent from the following description. In this specification, for purposes of explanation, numerous specific details of certain examples are set forth. References to "example" or similar language in this specification mean that a particular feature, structure, or characteristic described in connection with that example is included in at least one example, but not necessarily in other examples. It should also be noted that some examples are described schematically, in which certain features are omitted and / or certain features must be simplified to facilitate explanation and understanding of the concepts on which the examples are based.

[0018] In the examples described herein, at least a portion of a first neural network layer is used to generate output data. A key associated with that at least portion of the first neural network layer is also generated and stored in a storage device. The key is retrieved from the storage device, and a version of the output data input to a second neural network layer is obtained. Using this key, it can then be determined whether this version of the output data differs from the output data generated using the first neural network layer. This method, for example, allows detection of whether the output data has been tampered with by a malicious party, for example, seeking to expose information (such as confidential or other sensitive information) available in the neural network. In this way, the security of the neural network including the first neural network layer can be improved. This method can also, for example, allow detection of whether the output data has been corrupted, for example, due to a soft error, such as an error (e.g., a broken component) that causes stored data (such as output data) to change in an unexpected manner (e.g., due to memory corruption). The soft error may occur, for example, in the storage device, or it may occur in the logic circuitry of a processor configured to perform the method according to the examples described herein. The soft error may, for example, be caused by a cosmic ray colliding with the processor.

[0019] Figure 1 This is a schematic diagram illustrating a method 100 for determining whether output data 101 generated by the first neural network layer 104 of neural network 102 is different from the version of output data 103 used as input to the second neural network layer 106 of neural network 102. See reference... Figure 4 In a more detailed description, method 100 may be executed using at least one processor, such as a neural processing unit (NPU).

[0020] The method in this paper uses neural networks (such as, Figure 1The neural network (102) can be of various types, such as convolutional neural networks (CNNs), recurrent neural networks (RNNs), etc. Generally, a neural network typically consists of several interconnected neurons forming a directional weighted graph, where vertices (corresponding to neurons) or edges (corresponding to connections) are associated with weights and / or biases, respectively. The weights and / or biases can be adjusted for a specific purpose throughout the training of the neural network, thereby changing the output of individual neurons and thus changing the overall output of the neural network. Neurons can be arranged into layers such that information can flow from a given neuron in one layer of the neural network to one or more neurons in another layer. In this way, the output of a given layer can serve as the input to the next layer, and this process can continue for subsequent layers in the neural network. Neural networks such as CNNs can include various layer types, such as convolutional layers, pooling layers, activation layers, and fully connected layers. Layers can be fused (combined) and processed as fused layers. Other layer types are also known, and various subcategories exist. Generally, a particular neural network is defined by the specific arrangement and combination of layer types within that particular neural network.

[0021] In method 100, output data 101 is generated using a first neural network layer 104. For example, output data 101 can be generated by applying operations to input data 105 input to the first neural network layer 104 using weights associated with it. Input data 105 is data to be processed by the neural network 102, such as image data, audio data, or text data. The operations applied by the first neural network layer 104 can be, for example, a convolution of the input data 105 and the weights, and in some cases, a summation of biases. In some examples, the data generated by such operations can also be processed, for example, by applying the output of activation functions and / or pooling operations. In this way, the first neural network layer 104 can be a fusion of various types of layers, such as convolutional layers, pooling layers, activation layers, and / or fully connected layers. In some examples, input data 105 can be divided into multiple parts, such that input data 105 is processed piece by piece using the first neural network layer 104, for example, piece by piece. The same principle can also be applied to the second neural network layer 106.

[0022] The internal storage device 112 of the data processing system 107 configured to implement the neural network 102 may not have sufficient storage capacity to store all the data associated with the use of the neural network 102, such as input data 105, output data 101, and weight data representing the weights associated with the corresponding layers of the neural network 102. Therefore, in Figure 1 In method 100, the output data 101 generated using the first neural network layer 104 is stored in an external storage device 108, which in this example is external to the data processing system 107. (See reference...) Figure 4 A more detailed description of what can be used as Figure 1 An example of a data processing system 107 is provided. In this example, the output data 101 generated using the first neural network layer 104 can be compressed. This reduces the size of the output data 101, allowing for more efficient use of storage capacity. For example, the output data 101 can be compressed before being stored in external storage device 108. This means that a lower capacity of external storage device 108 can be used to store the output data 101. Lossy or lossless compression can be used to compress the output data 101. Any suitable lossy or lossless compression algorithm can be used to compress the output data 101. However, in other examples, it is not necessary to send the output data 101 to external storage device 108. Instead, the output data 101 can be stored in a storage device such as... Figure 1 The data processing system 107 is stored in the internal storage device 112 of the data processing system (for example, if the internal storage device 112 has sufficient storage capacity).

[0023] exist Figure 1 In method 100, key generator 110 generates key 109 associated with the first neural network layer 104. The association between key 109 and the first neural network layer 104 is determined by... Figure 1The double-headed arrow 119 is shown in the diagram. In some cases, information may be sent from the first neural network layer 104 to the key generator 110. For example, an instruction instructing the key generator 110 to generate key 109 may be sent from the first neural network layer 104, and / or output data 101 may be sent to the key generator 110 for generating key 109. In other examples, the data processing system 107 may be configured differently to still cause the key generator 110 to generate key 109 in response, for example, after processing input data 105 using the first neural network layer 104. In the examples described herein, the key is, for example, an encryption key, which may be in the form of a random sequence of characters generated by a random number generator or a pseudo-random number generator. In other examples, a key derivation function may be used to determinely derive the key, which may use a cipher or passphrase to derive the key. Keys may be generated on a per-layer basis. Thus, the key 109 associated with the first neural network layer 104 will be different from the key associated with the second neural network layer 106. In some examples, key 109 is unique for the first neural network layer 104. This reduces the predictability of key 109 for malicious actors. A new key can be generated for each run of the first neural network layer 104. For example, whenever output data 101 is generated using the first neural network layer 104, key generator 110 can generate a new key 109 associated with the first neural network layer 104. Thus, multiple runs of the first neural network layer 104 at different times will result in different keys being generated each time. Introducing variability into the keys generated based on each layer and each run reduces the risk of key determination and also limits the amount of information leaked. In the example where lossless compression is applied to output data 101, key generator 110 can generate key 109 before or after compression. However, in the example where lossy compression is applied to output data 101, key generator 110 can generate key 109 after lossy compression. Figure 1 In method 100, the key 109 generated by key generator 110 is stored in the internal storage device 112 of data processing system 107. In other examples, key generator 110 may generate multiple keys per layer. This can further improve the security of data processing system 107 by making it more difficult for a malicious party to determine the key associated with a given portion of the data. For example, output data 101 may be divided into multiple parts, such that for each part of output data 101 generated using the first neural network layer 104, key generator 110 generates a key.

[0024] exist Figure 1 In method 100, data analyzer 114 obtains key 109 from internal storage device 112. In this example, data analyzer 114 also obtains a version of output data 103 for input into the second neural network layer 106 from external storage device 108.

[0025] exist Figure 1 In the example, the neurons of neural network 102 are arranged in layers such that the output of a given layer can serve as the input to the next layer. For example, in the case where method 100 is used to process data representing an image, the first neural network layer 104 and the second neural network layer 106 can be corresponding layers of a convolutional neural network. In this example, the output data 101 can be an output feature map, which can be used as an input feature map for input to the second neural network layer 106. In such cases, Figure 1 Method 100 can be executed by a processor in an image processing system.

[0026] The version of output data 103 used as input to the second neural network layer 106 should be the same as the output data 101 generated using the first neural network layer 104, provided that the version of output data 103 has not been altered. However, the version of output data 103 used as input to the second neural network layer 106 may differ from the output data 101 generated using the first neural network layer 104, for example, if a malicious party has modified the version of output data 103. For example, this version of output data 103, and in some cases the weights associated with the second neural network layer 106, may be available from external storage device 108 before processing this version of output data 103 using the second neural network layer 106. In the example where output data 101 is compressed, this version of output data 103 may be decompressed before further processing of this version of output data 103 (e.g., using the second neural network layer 106). A malicious actor could gain unauthorized access to external storage device 108 and could alter the output data 101 generated by the first neural network layer 104 and stored in external storage device 108 to generate a version of output data 103 for input to the second neural network layer 106, different from the output data 101 originally stored in external storage device 108. The malicious actor could modify the output data 101 to extract information characterizing the second neural network layer 106. For example, the output data 101 could be replaced or modified to provide oriented data, such as an impulse response function, for input to the second neural network layer 106 as the version of output data 103. This would allow the second neural network layer 106 to process the oriented data (e.g., in the case of a convolutional layer, by convolving the weights associated with the second neural network layer 106 with the oriented data) to output the weights of the second neural network layer 106 itself. This would allow a malicious actor to determine the weights of the second neural network layer 106, thereby exposing the weights to manipulation or unauthorized use. The security of neural network 102 is improved by determining whether the output data 101 generated by the first neural network layer 104 has been tampered with or corrupted before the version of output data 103 is input into the second neural network layer 106.

[0027] Many applications of neural networks, such as facial recognition, require that the weights associated with the neural network be secure and not exposed to any other applications. In an example of using facial recognition to unlock a smartphone belonging to a specific user, exposure of the weights could allow a third party who knows these weights to input specific predetermined data, which could incorrectly detect the specific user and unlock the smartphone when manipulated through the weights. The method presented in this paper allows for the identification of unauthorized modifications to the output data 101. Therefore, appropriate mitigation actions can be taken to reduce the risk of sensitive data, such as neural network weights, being exposed.

[0028] Using key 109, data analyzer 114 determines whether the version of output data 103 differs from the output data 101 generated using the first neural network layer 104. In this way, data analyzer 114 can determine whether output data 101 has been modified or replaced by different data since its generation and before being input into the second neural network layer 106. In the example where key generator 110 generates multiple keys per layer, the version of output data 103 for which this determination is performed can represent a smaller portion of the data because, in this case, output data 101 generated using the first neural network layer 104 can be divided into multiple parts, allowing key generator 110 to generate a key for each part of output data 101. This reduces the processing power required to perform the determination.

[0029] Data analyzer 114 may respond in a specific manner to determining whether the version of output data 103 used as input to the second neural network layer 106 differs from the output data 101 generated using the first neural network layer 104. For example, data analyzer 114 may determine that the version of output data 103 is the same as output data 101, and therefore output data 101 has not been modified or replaced since it was first generated using the first neural network layer 104. The version of output data 103 can then be processed by the second neural network layer 106 without the risk of exposing the weights and biases of the second neural network layer 106. Therefore, Figure 1 In the example, data analyzer 114, in response to such a determination, sends a version of output data 103 to a second neural network layer 106 for processing.

[0030] In this example, in response to determining that the version of output data 103 differs from output data 101, data analyzer 114 generates data indicating that the version of output data 103 differs from output data 101. The data generated by data analyzer 114 may, for example, indicate that output data 101 has been modified or replaced since it was first generated using the first neural network layer 104. This indicates, for example, that a malicious party may be attempting to input malicious data, such as targeted data, into the second neural network layer 106. In this example, the data indicating that the version of output data 103 differs from output data 101 can be used to prevent the processing of the version of output data 103 using the second neural network layer 106, thereby reducing the risk of exposure of the weights associated with the second neural network layer 106. The data indicating that the version of output data 103 differs from output data 101 may, for example, represent an indication of such a difference, and this data may be in any suitable format, such as indicating whether output data 103 is a different binary flag. Such an indication may be processed by the second neural network layer 106 to determine whether to continue processing the version of output data 103. However, in other cases, the data analyzer 114 may optionally or additionally generate appropriate control instructions for sending to the second neural network layer 106, instructing the second neural network layer 106 not to process the version of output data 103. This reduces the risk that a malicious party could successfully extract the weights associated with the second neural network layer 106 by tampering with the output data 101. It should be understood that this can be repeated for subsequent neural network layers. Figure 1 Method 100 reduces the risk of weights associated with subsequent neural network layers being exposed.

[0031] After determining that the version of output data 103 differs from that of output data 101, various actions can be taken. For example, data indicating that the version of output data 103 differs from that of output data 101 can be used to control the second neural network layer 106 to omit processing of the version of output data 103 and / or further halt processing performed by the remaining layers of neural network 102. In another example, for instance, if at least one processor of data processing system 107 is controlled and configured by a central processing unit (CPU), an interrupt command can be sent to the CPU in response to determining that the version of output data 103 differs from that of output data 101. In another example, at least one processor of data processing system 107 (e.g., an NPU) can stop processing data to reduce the risk of processing malicious data. In some examples, at least one processor (e.g., an NPU) can be reset.

[0032] There are various ways to determine whether the version of the output data is different from the original output data. Figure 2 This is a schematic diagram illustrating how the determined method 200 is performed. (and) Figure 1 The corresponding feature is similar Figure 2The features are marked using the same reference numerals, but with an additional 100; the corresponding descriptions will be applied.

[0033] exist Figure 2 In method 200, the first signature calculator 116 calculates the first signature 111 based on the key 209 generated by the key generator 210 and the output data 201 generated by processing the input data 205 using the first neural network layer 204 of the neural network 202.

[0034] In the examples described herein, the signature can be a value calculated from data and a key, which can be used to verify that the data has not been modified or replaced since its generation. In one example, the signature can be a Cyclic Redundancy Check (CRC) based on the remainder of a polynomial division of the data by the value determined by the key. In the case that the first signature 111 is a CRC, this can be calculated based on the remainder of the output data 201 generated by the first neural network layer 204 divided by the remainder of a polynomial division by the key 209 associated with the first neural network layer 204. Although CRC is given here as an example, it should be understood that any type of signature (e.g., a hash-based signature) can be used to verify that the data has not been modified or replaced since its generation. Because the key can be generated based on each layer and / or each run as described above, it should be understood that the signature can differ for the outputs of different neural network layers and / or the outputs of the same neural network layer at different times. The variability of the signature calculated based on each layer and each run reduces the risk of the signature being determined. Figure 2 In method 200, the first signature 111 calculated by the first signature calculator 116 is stored in the internal storage device 212 of the data processing system 207.

[0035] exist Figure 2In the example shown, data analyzer 214 includes a second signature calculator 118 that calculates a second signature 113 based on a key 209 obtained from internal storage 212 and a version of output data 203 used as input to a second neural network layer 206 of neural network 202. This version of output data 203 is obtained from external storage 208 outside of data processing system 207. The second signature 113 can be calculated using the same method used to calculate the first signature 111, but based on the key 209 and a version of output data 203 (instead of the key 209 and output data 201). For example, if the first signature 111 is a CRC, the second signature 113 can also be a CRC, but instead of the first signature 111, it is calculated based on the remainder of a polynomial division of the version of output data 203 (instead of the output data 201 generated by the first neural network layer 204 in the case of the first signature 111) with the key 209 associated with the first neural network layer 204. In this scenario, to determine whether a version of output data 203 differs from output data 201, the data analyzer 214 includes a signature comparator 120. The signature comparator 120 determines whether a second signature 113 is equal to a first signature 111 obtained from internal storage 212. Thus, determining that the second signature 113 is not equal to the first signature 111 may indicate that the output data 201 generated using the first neural network layer 204 has been modified or replaced by potentially malicious data. Therefore, a check can be performed before processing a version of output data 203 using the second neural network layer 206, for example, to check whether the weights associated with the second neural network layer 206 are at risk of exposure. In this example, in response to the signature comparator 120 determining that the second signature 113 is equal to the first signature 111, only the second neural network layer 206 can be used to process the version of output data 203. The second signature 113 being equal to the first signature 111 may indicate that the output data 201 generated using the first neural network layer 204 has not been modified or replaced by malicious data such as targeted data. This reduces the risk of the weights associated with the second neural network layer 206 being exposed.

[0036] Figure 3 This is a schematic diagram illustrating a method 300 for determining whether the version of output data 303 differs from that of output data 301, based on another example. Figure 1 The corresponding feature is similar Figure 3 The characteristic parts are marked using the same reference numerals, but with an additional 200; corresponding descriptions will be applied. It should be understood that... Figure 3 Method 300 can be applied alone or in combination with... Figure 2Method 200 is combined with an application to determine whether the output data 301 generated using the first neural network layer 304 of the neural network 302 is different from the version of the output data 303 obtained in this case from an external storage device 308 outside the data processing system 307 used to implement the neural network 302.

[0037] exist Figure 3 In method 300, encryption engine 122 encrypts output data 301 generated by processing input data 305 using a first neural network layer 304. Encryption engine 122 uses a key 309 generated by key generator 310 to encrypt the output data 301. The encryption engine may include or use at least one processor configured to implement an encryption scheme to encrypt data. This at least one processor may be a reference... Figure 4 A more detailed description of the NPU processor. For example, encryption engine 122 may implement an encryption scheme such that output data 301 is encrypted to generate encrypted output data 115 stored in external storage device 308. The encryption engine may be configured to encrypt output data 301 using a specific encryption algorithm. The encryption algorithm implemented by the encryption engine may differ for the output of different neural network layers. In other words, encryption engine 122 may use different corresponding encryption algorithms to encrypt data received from different corresponding neural network layers. Furthermore, as mentioned above, key 309 may be generated based on each layer and / or each run. Such variability in key 309 and / or encryption algorithm reduces the risk that a malicious party could successfully decrypt encrypted output data 115. In this example, key 309 is stored in internal storage device 312, which is generally more secure than external storage device 308.

[0038] Storing the encrypted output data 115 in external storage device 308 (which is typically more vulnerable to unauthorized access than internal storage device 312) instead of the output data 301 reduces the risk of a malicious party obtaining a usable version of the output data 301. For example, even if a malicious party obtains the encrypted output data 115 from external storage device 308, they must decrypt the encrypted output data 115 before using it, which is typically difficult (if possible) without the key 309. This can be beneficial in certain use cases involving the processing of sensitive input data 305, such as super-resolution neural networks. Super-resolution neural networks can be used to generate high-resolution images from their low-resolution counterparts, and it may be necessary for the inputs and outputs associated with neural network 302 (e.g., the inputs and outputs associated with a given neural network layer) to be secure and not exposed to any other applications. In the example of using a super-resolution network to amplify video content hidden behind a subscriber paywall, the exposure of the input (e.g., a low-resolution image) or output (e.g., a high-resolution image) could allow third parties to access the video content without paying the required subscription fee. However, the method described herein can be used to reduce the risk of private content being exposed to unauthorized parties. In some examples, a malicious party could replace or modify input data 305 to provide directional data, such as an impulse response function, for input into the first neural network layer 304. This would allow the first neural network layer 304 to process the input data 305 and output the weights and / or biases of the first neural network layer 304. However, if the output data 301 is encrypted, a malicious party would be unable to decrypt the encrypted output data 115 (and thus access the weights and / or biases) unless they gain access to the key 309, which is securely stored in internal storage 312. This reduces the risk of exposure of the weights and / or biases of the first neural network layer 304.

[0039] exist Figure 3 In method 300, decryption engine 124 uses key 309 obtained from internal storage device 312 to decrypt the version of output data 303 obtained from external storage device 308. Thus, decryption engine 124 generates a decrypted version of output data 117. Decryption engine 124 may include or use at least one processor configured to implement a decryption scheme to decrypt the data. This at least one processor may be a reference... Figure 4 A more detailed description of the NPU processor. The decryption engine 124 is configured to implement an appropriate decryption algorithm to decrypt the version of the output data 303 previously encrypted by the encryption engine 122. Figure 3In the example, a symmetric-key encryption algorithm is used, where the same key 309 is used to encrypt output data 301 and decrypt a version of output data 303. However, this is merely an example, and in other cases, an asymmetric-key encryption algorithm can be used alternatively. The version of output data 303 is decrypted by the decryption engine 124 before being fed into the second neural network layer 306.

[0040] In this example, data analyzer 314 determines whether the version of output data 303 differs from output data 301 based on the decrypted version of output data 117. For example, if the version of output data 303 has been modified, for instance, by a malicious party, decrypting the version of output data 303 would typically result in the decrypted version of output data 117 being unusable as input to the second neural network layer 306. Furthermore, if a malicious party has replaced the encrypted output data 115 stored in external storage device 308 with, for example, directional data used as the version of output data 303, the decrypted directional data will differ from the directional data that the malicious party intends to process using the second neural network layer 306 (with the aim of exposing the weights of the second neural network layer 306 as described above). Therefore, processing the decrypted directional data using the second neural network layer 306 will not output the weights associated with the second neural network layer 306 in the same way as processing the directional data of the initial input. In other examples, data analyzer 314 may evaluate the characteristics (which may be referred to as statistics) of the decrypted version of output data 117 to determine whether the version of output data 303 differs from output data 301. For example, data analyzer 314 can evaluate the mean or variance of a decrypted version of output data 117 or a portion of output data 117. If, for example, the version of output data 303 has been modified by a malicious party, the characteristics of the decrypted version of output data 117 may differ significantly from the expected characteristics, for example, based on the characteristics of output data 301. In response to data analyzer 314 determining that the characteristics of the decrypted version of output data 117 differ from the expected characteristics, data analyzer 314 can generate data indicating that the version of output data 303 differs from output data 301, as referenced above. Figure 1 As described in data analyzer 114. This allows data analyzer 314 to prevent further processing of the decrypted version of output data 303, for example, reducing the risk of weights associated with the second neural network layer 306 being exposed. For example, data analyzer 314 can determine whether the mean and / or variance of the decrypted version of output data 117 is outside a predetermined range, and in response to the mean and / or variance being outside the predetermined range, generate data indicating that the version of output data 303 is different from the output data.

[0041] Figure 4This is a schematic diagram of a data processing system based on an example. In this example, the data processing system is an image processing system 126 arranged to receive images as input and process the images. However, this is merely an example. It should be understood that this includes... Figure 4 The image processing system 126 may be implemented using similar or identical components of a data processing system, for example, to process other types of data, such as text data or audio data. The image processing system 126 may be implemented, for example, in hardware (e.g., in the form of a system-on-a-chip (SoC) that may be or include suitable programming circuitry), or in software, or a combination of hardware and software.

[0042] Image processing system 126 includes a neural processing unit (NPU) 128. NPU 128 is operable or otherwise configured to perform any of the methods described herein, such as Figure 1 , Figure 2 and / or Figure 3 Methods 100, 200, and / or 300. In other examples, the image processing system 126 may alternatively include any processor or group of processors that can be configured to implement a neural network and thus perform any of the foregoing methods. For example, a neural network such as that illustrated herein may be implemented using at least one of a neural processing unit (NPU), a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), and / or a coprocessor. An example is a CPU that includes a driver to provide an interface between the software configured to control or configure the neural network and the NPU 128.

[0043] The image processing system 126 also includes a storage device 130 for storing the key and, in some examples, storing the first signature as described in the examples above. The storage device 130 may be a static random access memory (SRAM). Alternatively, the storage device 130 may be or include multiple unit storage devices. Typically, a unit storage device is an electronic component having two stable states, for example, one stable state representing a value of zero and the other a value of one. Flip-flops and latches are examples of unit storage devices. When the key data size is relatively small, multiple unit storage devices may be used to store the key. The storage device 130 may be an on-chip or local storage device of the image processing system 126. In this case, the storage device 130 may correspond to... Figures 1 to 3 The internal storage devices 112, 212, and 312 are shown. Storing the key in local storage allows for faster access to the key and more secure key storage. In some cases, storage device 130 may include multiple separate storage areas.

[0044] Image processing system 126 includes a direct memory access (DMA) engine 132 to control data flow within image processing system 126. In this example, DMA engine 132 is configured to allow NPU 128 to directly access storage device 130, independent of other components inside and outside image processing system 126. DMA engine 132 can be configured to transfer keys only between NPU 128 and storage device 130 within image processing system 126, without transferring keys to other components, such as any other internal components of image processing system 126 and / or any components outside image processing system 126. For example, keys stored in storage device 130 can be accessed solely by the unit within image processing system 126 responsible for controlling data flow (i.e., DMA engine 132), which in turn can control which other components the keys are provided to. For example, DMA engine 132 can be configured to send keys to NPU 128, but not to other components of image processing system 126. This reduces the chance of a malicious party obtaining the keys, as a limited number of components within image processing system 126 have access to the keys. In an example where storage device 130 includes multiple unit storage devices such as multiple triggers, NPU 128 may not require DMA engine 132 to access storage device 130. Generally, it should be understood that in some examples, image processing system 126 is configured such that storage device 130 can only be accessed by NPU 128 of image processing system 126.

[0045] The image processing system 126 is configured to implement Figure 2 In the case of method 200, storage device 130 can also store data such as... Figure 2 The first signature is calculated by the first signature calculator 116. In the same manner as described above for the key, the first signature stored in storage device 130 can also be accessed only by the unit in image processing system 126 responsible for controlling the data flow (i.e., DMA engine 132), which in turn can control which other components the first signature is provided to. For example, DMA 132 can be configured to send the first signature to NPU 128, but not to other components inside and / or outside image processing system 126. Therefore, the first signature can have access restrictions similar to those on the key. This similarly reduces the chance of a malicious party obtaining the first signature.

[0046] Image processing system 126 includes interface 134 through which NPU 128 communicates with external storage device 136 in a manner controlled by DMA engine 132. External storage device 136 is external to image processing system 126 and may be random access memory (RAM), such as DDR-SDRAM (Double Data Rate Synchronous Dynamic Random Access Memory). In other examples, external storage device 136 may be or include non-volatile memory, such as read-only memory (ROM) or solid-state drive (SSD), such as flash memory. External storage device 136 in the examples may include additional storage devices, such as magnetic media, optical media, magnetic tape media, optical discs (CDs), digital versatile discs (DVDs), or other data storage media. For example, external storage device 136 may be or include storage devices (such as main memory or system memory) arranged for a computing system or device to process input image data using image processing system 126. Such computing devices may be personal computers, smartphones, tablets, image capture devices (such as cameras or camcorders), or in-vehicle computer devices that can be coupled to or installed in a vehicle such as a car, but this is not intended to be a limitation.

[0047] As described above, when implementing a neural network, it may be impossible to store all data, including, for example, input data, output data, and data corresponding to the operations involved in the neural network, such as weights and / or biases, in the storage device 130 within the image processing system 126. At least some of the data may be stored in an external storage device 136, which may include volatile and / or non-volatile storage devices. The NPU 128 may access the external storage device 136 when executing the neural network via interface 134. In this example, the NPU 128 is configured to send output data to the external storage device 136 outside the image processing system 126. In this case, the NPU 128 is also configured to obtain a version of the output data from the external storage device 136, as described in the example above, which is used as input to a second neural network layer. Using the external storage device 136 in conjunction with the storage device 130 of the image processing system 126 allows for the storage of a larger amount of data associated with the neural network processed by the NPU 128, compared to using only the storage device 130 of the image processing system 126.

[0048] Figure 5 It includes Figure 4A schematic diagram of the internal components of an exemplary computing system 140 of the image processing system 126. The computing system 140 includes a display processor 144, which in this example is configured to process image data output by the image processing system 126 to generate an image for display. For example, the image processing system 126 may be configured to implement a super-resolution neural network, and the display processor 144 may be configured to process image data representing a high-resolution image output by the image processing system 126 to generate a high-resolution image for display.

[0049] In this example, the image processing system 126 is arranged to perform the reference. Figure 1 The method is described to facilitate the determination, using a key associated with the first neural network layer (which may be referred to as the first key in this example), of whether the output data generated by the first neural network layer (which may be referred to as the first output data in this example) differs from a version of the output data. The key is sent to a data processing system (such as...). Figure 4 and Figure 5 The method 152 of the image processing system 126 is shown in Figure 6 middle.

[0050] exist Figure 6 In method 152, item 154 includes a version that uses a second neural network layer to process the output data to generate second output data. (See above reference...) Figure 1 As discussed, in response to the image processing system 126 (e.g., the NPU of the image processing system 126) determining that the version of the first output data is the same as the first output data generated using the first neural network layer, a second neural network layer may be used to process the version of the first output data. In some examples, in this case, the second neural network layer may represent the last neural network layer of the neural network, such that the second output data represents the output of the neural network. In other examples, the second neural network layer may be another neural network layer of the neural network, such that the second output data represents the output of a neural network layer that is not the last neural network layer of the neural network.

[0051] Item 156 of method 152 includes generating a second key. In this case, the second key is associated with a second neural network layer. For example, the second key may be unique for the second neural network layer. Similar to the first key, the second key may also be generated on a per-run basis, such that multiple runs of the second neural network layer at different times will result in different keys being generated per run. In other examples similar to the example described above with reference to the first neural network layer, multiple keys associated with the second neural network layer may be generated.

[0052] Item 158 of method 152 includes sending the second output data to a second data processing system. In this example (where used...) Figure 5The computing system 140 (to implement method 152) can be considered as a first data processing system, and the image processing system 126 can be considered as a second data processing system. However, it should be understood that these are merely non-limiting examples of the first and second data processing systems, and Figure 6 Method 152 can be performed using other data processing systems besides these.

[0053] Item 160 of method 152 includes sending a second key to a second data processing system, which in this example is a display processor 144. The second output data and the second key may be sent to the second data processing system together, for example, in the same communication, or they may be sent separately, for example, at different corresponding times. (See reference again) Figure 5 , Figure 5 The computing system 140 includes a dynamic memory controller (DMC) 146, which controls access to a storage device 148 used by the image processing system 126 and the display processor 144. In this example, a second key is sent to the display processor 144 via the shared storage device 148, accessible by both the image processing system 126 and the display processor 144 through the DMC 146. This ensures the secure transmission of the second key to the display processor 144, reducing the risk of a malicious party obtaining the second key. Figure 5 As shown, when the second key is stored in shared storage device 148, the second key can only be accessed by the unit of computing system 140 responsible for controlling the data flow (i.e., DMC 146 in this example) (but this need not be the case in other examples). Storage device 148 storing the second key can be an external storage device (such as...) storing the first output data and / or the second output data. Figure 4 The external storage device 136 shown is the same as the external storage device (external to the image processing system 126). However, in other cases, the storage device 148 for storing the second key may be different from the storage device for storing data associated with the neural network (such as the first output data and / or the second output data). For example, Figure 5 The computing system 140 itself can be implemented in hardware, for example, as a SoC. In this example, the storage device 148 can be an internal storage device of the computing system 140, and the storage device for storing data associated with the neural network can be external to the computing system 140.

[0054] The transfer of the second key to the display processor 144 can be securely performed by assigning a trust level to each component of the computing system 140 to determine the degree of access that component has to certain data or other components, such as storage device 148 storing the second key. For example, a component within a secure environment (“or secure domain”) can be trusted within the computing system 140 and is therefore allowed access to, for example, security-sensitive data within the computing system 140. However, components outside the secure environment (e.g., in a less secure environment or “non-secure domain”) may not be allowed access to such security-sensitive data, such as the second key stored in storage device 148. Thus, a component within the secure environment may have access to certain storage devices (e.g., secure or “protected” memory areas) that are inaccessible to components outside the secure environment and the system. In this example, both the image processing system 126 and the display processor 144 can be assigned a trust level that indicates that the image processing system 126 and the display processor 144 are within a secure domain, thereby allowing them to access storage device 148. Therefore, this allows the second key to be securely sent to the display processor 144 using a shared storage device 148 accessible by both the image processing system 126 and the display processor 144. In this case, the shared storage device 148 can also be assigned a trust level indicating that it is within a security domain, thereby allowing it to store security-sensitive data such as the second key. These access controls can utilize technologies such as Arm... ® TrustZone ® This is implemented using embedded hardware technology. In this way, the second key can be shared only with components of the computing system 140, thereby reducing the risk of an unauthorized party obtaining the second key.

[0055] Figure 5 The computing system 140 uses a system bus 150. This allows data to be transferred between various components, enabling both the image processing system 126 and the display processor 144 to access the shared memory device 148 via the DMC 146. The bus 150 can be or include any suitable interface or bus. For example, an ARM processor can be used. ® Advanced Microcontroller Bus Architecture (AMBA) ® Interfaces, such as Advanced Extensible Interface (AXI).

[0056] The second output data can be encrypted before being sent to the display processor 144, for example, using methods such as those described above. Figure 3 The encryption engine 122 is used for encryption. The second output data can be encrypted by applying an encryption algorithm using a second key, for example, using a method similar to encryption. Figure 3 The output data is processed in the manner of 301.

[0057] Display processor 144 can use a second key to determine whether the version of the second output data obtained by display processor 144 from image processing system 126 is different from the second output data generated using the second neural network layer of image processing system 126. This determination can be used with reference to... Figure 2 and Figure 3 The methods described are similar to those used to determine whether the version of the output data differs from the original output data. In the example of calculating the first and second signatures, the first signature would be calculated by the image processing system 126, for example, by its NPU, based on the second key and the second output data. Then, because, as described above, the signature and key can have the same accessibility requirements, the first signature can be sent to the display processor 144 in a manner similar to how the second key is sent. The display processor 144 can be configured to calculate the second signature based on the version of the obtained second output data and the second key, such that it can be determined whether the second signature is equal to the first signature. This allows the display processor 144 to identify whether the second output data has been tampered with or corrupted since it was generated using the second neural network layer of the image processing system 126.

[0058] The display processor 144 can then determine whether to process the second output data based on whether the version of the second output data differs from the second output data. For example, the display processor 144 can process only the version of the second output data if it determines that the version of the second output data is the same as the second output data. This reduces the risk of exposing sensitive information to malicious parties.

[0059] In other examples, such as reference Figure 5In cases where the image processing system 126 and the display processor 144 are within different computing systems, it may be impossible to send the second key to the display processor 144 using shared storage accessible to both the image processing system 126 and the display processor 144. In these examples, the second key can be securely transmitted to the display processor 144 by sending it through a first data channel, which is different from the second data channel through which the second output data is sent to the display processor 144. In this example, the data channel describes the information route taken between the two components (e.g., the image processing system 126 and the display processor 144). Therefore, individual data channels can vary in terms of their bandwidth, communication protocols, the devices through which information is routed, and / or various other factors. For example, the first data channel for sending the second key to the display processor 144 may use a cryptographic communication protocol such as the Diffie-Hellman key exchange protocol to securely exchange the second key between the NPU and the display processor 144. Sending the second key to the display processor 144 through a separate secure data channel reduces the risk of a malicious party obtaining the second key.

[0060] Figure 7 This is based on other examples, including Figure 4 A schematic diagram of the internal components of the computing system 162 of the image processing system 126. The computing system 162 includes an image signal processor (ISP) 166, which may be, for example, the ISP of a digital camera. The ISP processes raw data captured by the image sensor of the digital camera to generate image data, and then sends the image data to the image processing system 126 for further processing. For example, the image data may be input into a first neural network layer implemented by the image processing system 126.

[0061] In this example, the image processing system 126 is arranged to perform the reference. Figure 1 The method is described in order to use a key associated with the first neural network layer (which may be referred to as the first key in this example) to determine whether the output data generated by the first neural network layer (which may be referred to as the first output data in this example) is different from the version of the output data. Figure 8 The diagram illustrates a method 174 for determining whether the input data used to input into a first neural network layer is different from a version of the input data used to input into the neural network, for example, the first neural network layer implemented by the image processing system 126.

[0062] Figure 8Method 174 provides a way to determine whether the input data for a first neural network layer implemented by the image processing system 126 (e.g., by the NPU of the image processing system 126) has been tampered with to improve the security of the neural network. As described above, if the data used as input to the first neural network layer has been replaced or modified by malicious data such as directional data, the operation of the first neural network layer on that malicious data may expose the weights of the first neural network layer.

[0063] Item 176 of method 174 includes obtaining a third key from a third data processing system, which in this example is ISP 166. Figure 7 The image processing system 126 can be considered as corresponding to the first data processing system. Figure 8 Method 174 can be implemented in a computing system that includes at least one additional data processing system (such as a second data processing system, e.g., a display processor), but is not required to do so. It should be understood that these are merely non-limiting examples of the first, second, and third data processing systems, and Figure 8 Method 174 can be executed using other data processing systems besides these.

[0064] The third key obtained at item 176 of method 174 is associated with input data that is fed into the first neural network layer for processing by the image processing system 126 (e.g., by the NPU of the image processing system 126). This input data may be image data generated by the ISP 166, such that the third key is associated with the image data. The third key can be transmitted between the ISP 166 and the image processing system 126 using any of the aforementioned security mechanisms. For example, as... Figure 7 As shown, computing system 162 includes a dynamic memory controller (DMC) 168, which can be used to control access to storage device 170 for image processing system 126 and ISP 166. In this case, a third key can be sent from ISP 166 to image processing system 126 using shared storage device 170, which can be accessed by both image processing system 126 and ISP 166 via DMC 168. This can be done in a manner similar to that described in reference [reference missing]. Figure 5 The method described for transmitting the second key to the display processor 144.

[0065] Item 178 of method 174 includes obtaining a version of the input data to be input into the first neural network layer. The version of the input data is obtained, for example, by the image processing system 126 from the ISP 166. The version of the input data may be received directly from the ISP 166 by the image processing system 126, or it may be stored in a storage device (which may be the same as or different from the storage device 170 of the computing system 162) and subsequently obtained by the image processing system 126.

[0066] Item 180 of method 174 includes using a third key to determine whether the version of the input data differs from the input data. Therefore, if the image processing system 126 processes the version of the input data using the first neural network layer, it can be determined whether there is a risk of exposure to the weights associated with the first neural network layer. This determination can be performed using methods similar to those described above. In the example of calculating the first and second signatures, the first signature can be calculated by the ISP 166 based on the third key and the input data. The first signature can then be securely sent to the image processing system 126 (e.g., the same secure method as sending the third key to the image processing system 126). The image processing system 126 can then calculate the second signature based on the obtained version of the input data and the third key, such that it can be determined whether the second signature is equal to the first signature, and thus whether the input data has been tampered with during transmission between the ISP 166 and the image processing system 126.

[0067] Image processing system 126 may process the version of input data using a first neural network layer in response to determining that the version of input data is the same as the input data. In response to determining that the version of input data is different from the input data, image processing system 126 may be configured to generate data indicating that the version of input data is different from the input data, and / or image processing system 126 may omit the processing of the version of input data using the first neural network layer. This reduces the risk of the first neural network layer being used to process input data containing malicious data (e.g., malicious data that may expose the weights associated with the first neural network layer).

[0068] In some examples, the input data is encrypted using a third key to generate encrypted input data for transmission to the image processing system 126. In these examples, the image processing system 126 (e.g., the NPU of the image processing system 126) may include a decryption engine configured to decrypt a version of the input data to be input into the first neural network using the third key. The decryption engine may, for example, be similar to the one referenced above. Figure 3The decryption engine 124 is the same as or similar to the one described above. Encrypting the input data in this way improves the security of the input data. In this example, the image processing system 126 can determine whether the version of the input data differs from the original input data based on the decrypted version of the input data.

[0069] although Figure 7 The computing systems include the ISP 166, but it should be understood that other systems similar to it... Figure 7 The computing system 162 may alternatively include different data processing systems, such as data processing systems for input devices other than image sensors (e.g., input devices including one or more sensors). In such cases, one or more sensors may be configured to acquire sensor data and send the sensor data to the image processing system 126 for processing. Multiple neural network layers implemented by the image processing system 126 may be used to perform tasks on the input data obtained from the input device, such as, for example, facial recognition where the data used for input to the image processing system 126 includes image data. Additionally or alternatively, the input device may include a decoder configured to output decoded data to the image processing system 126. For example, the decoder may be a video decoder that decodes video and outputs the decoded video to the image processing system 126 for processing by a neural network. It should be understood that the data sent from the input device to the image processing system 126 may be generated by the input device or may be obtained by the input device, for example, from another system or device. For example, the data used for input to the image processing system 126 may be image data generated by a graphics processing unit (GPU).

[0070] Therefore, it can be seen that the method described herein allows for the determination of whether the data used as input to a neural network layer has been altered since its initial generation. Furthermore, the method described herein reduces the risk of input data, output data, and / or data corresponding to operations involved in the neural network (e.g., weights and / or biases) being exposed to malicious actors. The security of input and output data here is not limited to the input and output of the neural network as a whole, but also applies to the input and output of individual neural network layers.

[0071] although Figure 5 and Figure 7 The computing systems 140 and 162 are shown as a single system (e.g., corresponding to a single device), but it should be understood that they are otherwise similar. Figure 5 and Figure 7 The computing systems 140 and 162 can be implemented as distributed systems. For example, Figure 5 The image processing system 126 and the display processor 144 can be located in different computing systems. Similarly, Figure 7The image processing system 126 and ISP 166 can be located within different computing systems. An example of a distributed system used in this paper is... Figure 9 The diagram illustrates a system 182 comprising a first computing system 184 and a second computing system 186 communicating with each other via a data communication network 188. For example, such as... Figure 7 A data processing system of the ISP 166 may be included in the first computing system 184, which in this example may be a digital camera. Another data processing system, such as image processing system 126, may be included in the second computing system 186, which in this example may be a laptop computer (although this is merely an example). In this case, the image processing system 126 in the laptop computer may be configured to implement a super-resolution neural network to upscale images received from the ISP of the digital camera of the first computing system 184 into image data. The data communication network 188 may include dedicated network links and / or public network links, and may include multiple interconnected networks. The first computing system 184 and the second computing system 186 may have integrated or externally coupled wired networking capabilities. Optionally or additionally, the first computing system 184 and the second computing system 186 may utilize wireless telecommunications systems (such as wireless telecommunications systems using the Long Term Evolution (LTE) standard) and have integrated or externally coupled wireless networking capabilities. One or both of the first computing system 184 and the second computing system 186 may be connected to one or more networks, including servers, routers, and other networked devices communicating using the Internet Protocol (IP). In this example, a third key can be securely transmitted from the ISP of the digital camera of the first computing system 184 to the data processing system of the laptop computer of the second computing system 186 via a first data channel, distinct from the second data channel through which image data obtained by the camera is transmitted to the laptop computer. For example, the third key can be transmitted via a separate secure data channel. This reduces the risk of a malicious party obtaining the third key. As explained with reference to the example above, this therefore reduces the risk of the input data and / or weights of the neural network implemented by the image processing system 126 of the laptop computer being exposed.

[0072] Consider another example. For a given neural network layer, such as those described above, it may not be possible to process all the input data using a single neural network layer. For instance, the entire input image or the input feature map generated by operations on the input image by one or more neural network layers may be too large to be processed by a single neural network layer. Therefore, in some examples, neural network layers can be used to divide the data input to the neural network layer into parts to be processed, such as sequentially or in parallel.

[0073] Apply this principle Figure 4 Image processing system 126, using output data generated by a first neural network layer, can partially send to external storage device 130, and a version of the output data can be partially obtained from external storage device 130. The order in which the portions of the output data are sent and the portions of the output data versions are obtained can be controlled by DMA engine 132. In some examples, the order in which portions of the output data are sent to external storage device 130 may be the same as the order in which the corresponding portions of the output data versions are obtained from external storage device 130. However, in other examples, the order in which portions of the output data are sent to external storage device 130 may be different from the order in which the corresponding portions of the output data versions are obtained from external storage device 130. Image processing system 126 (e.g., NPU 128 of image processing system 126) can be configured to determine whether a portion of the output data version is different from a corresponding portion of the output data based on a location identifier. In this case, the location identifier identifies the corresponding portion of the output data within the output data. Therefore, the determination of whether the output data has been tampered with can be performed sequentially, wherein the location identifier can be used to identify the portion of the output data from which the determination is being performed. In this example, the image processing system 126 (e.g., NPU 128) can be further configured to modify the key based on a location identifier and use the modified key to determine whether that portion of the output data differs from the corresponding portion of the output data. Thus, the key used for determination may be different for each portion of the output data, thereby providing additional difficulty for a malicious party to obtain each key associated with each portion of the output data. See below for reference. Figure 10 Provide an example of this situation.

[0074] Figure 10 It is an image processing system to be used according to the examples in this article (such as...) Figure 4 A schematic diagram of image 190 processed by image processing system 126 (as described above). In some examples, image 190 may be a feature map generated by processing data using one or more neural network layers. However, in other cases, image 190 may include a pixel array having corresponding pixel values ​​indicating characteristics (e.g., intensity and / or color) of corresponding portions of the scene represented by the image. In this example, image 190 includes four patches: upper left patch 192, upper right patch 194, lower left patch 196, and lower right patch 198. Although Figure 10The regions of image 190, not shown but represented by four tiles, may partially overlap. Image 190 may be represented by image data that can be divided into four parts, each part of the image data representing one of the four tiles of image 190. In this example, to explain the part-by-part processing, the image data representing image 190 is the output data generated using a first neural network layer, and the version of the output data represents the version of the image data used as input to the second neural network layer as described above.

[0075] In this example, the location identifier can be a unique identifier for each tile in image 190. In a simple example where image 190 is divided into four tiles, the location identifier can be a 2-bit binary number. For example, the portion of image data representing the upper left tile 192 can have a location identifier with a value of 00. The portion of image data representing the upper right tile 194 can have a location identifier with a value of 01. The portion of image data representing the lower left tile 196 can have a location identifier with a value of 10, and the portion of image data representing the lower right tile 198 can have a location identifier with a value of 11. In this way, the location identifier identifies the corresponding portion of image data within the image data.

[0076] In this example, the key associated with the first neural network layer is modified based on a location identifier. For example, the location identifier value can be appended to the key value to generate a modified key for each patch of image 190. Thus, for a given patch, the modified key associated with that patch will be used to determine whether a portion of the image data version differs from the corresponding portion of the image data. In this example, the storage device 130 of the image processing system 126 can still store the key without any appended location identifiers, allowing a single key to be stored instead of four different keys, thereby reducing the capacity requirement of the storage device 130. In this case, the location identifier of the given patch is appended to a single key because, and when the modified key for a given patch is needed, it is appended to the key.

[0077] To understand this principle, this example considers determining whether a portion of the image data version differs from the corresponding portion of the image data based on the signature. However, it should be understood that the same principle can be applied when determining the decrypted portion based on the image data version.

[0078] In this example, for a portion of image data, a first signature is calculated based on that portion of image data and a modified key generated by appending a location identifier of a given tile (e.g., the top-left tile 192) to a key associated with the first neural network layer. Storage device 130 can then store the unmodified key and the first signature. This portion of the image data can be sent to external storage device 136 via interface 134. This can be controlled by DMA engine 132. When the corresponding portion of a version of image data representing a version of the image (e.g., the top-left tile of a version of the image) is obtained from external storage device 136 using DMA engine 132, a matching location identifier can be appended to the key obtained from storage device 130 to recreate the modified key. The NPU 128 of the image processing system 126 is configured to calculate a second signature based on the reconstructed modified key and a portion of the version of the image data, such that NPU 128 can determine whether the second signature is equal to the first signature. In this way, the examples described herein can be applied one by one. In an alternative implementation, a separate key (and, in some examples, a separate signature) may be generated for each tile of image 190, thereby providing variability in the key generated for each tile.

[0079] Another example is envisioned. For instance, although the first neural network layer and the second neural network layer are referred to in the singular form in the examples described herein, it should be understood that either or both of the first and second neural network layers can be a fusion of multiple neural network layers, such that the processing of the neural network layers described above can actually involve the processing of multiple layers. For example, Figure 4 The NPU 128 of the image processing system 126 can be configured to generate output data by processing input data using at least one additional neural network layer before the first neural network layer, such that the output data is the output of the processing of multiple layers. This may mean that the determination of whether data generated by the neural network layers before being input into subsequent neural network layers may not be performed between each layer, but rather alternatively applied between groups of layers at each stage. In another non-limiting example, it should be understood that either or both of the first and second neural network layers may be sub-units of a layer. This may be the case for particularly large neural network layers, where at least one processor of the data processing system may not have the capability to implement the entire layer at a given time.

[0080] In the example above, the first neural network layer is used to generate output data, and a key associated with the first neural network is also generated. However, it should be understood that in some cases, at least a portion of the neural network layer (e.g., less than the entire neural network layer) can be used to generate the output data. In such cases, the key can be associated with at least a portion of the neural network layer. For example, a portion of the first neural network layer 104 can be used to generate... Figure 1 The output data 101 can be used to generate a key associated with that portion of the first neural network layer 104. This may involve generating multiple sets of output data, each associated with a different corresponding portion of the neural network layer (e.g., the first neural network layer). Multiple sets of keys can then be generated, each associated with a different corresponding portion of the neural network layer.

[0081] It should be understood that any feature structure described with respect to any one example may be used alone or in combination with other feature structures described, and may also be used in combination with one or more feature structures of any other example, or in combination with any feature structure of any other example. Furthermore, equivalents and modifications not described above may be employed without departing from the scope of the appended claims.

[0082] According to the present invention, a data processing system is provided, comprising: a storage device; and at least one processor, the at least one processor being configured to: generate output data using at least a portion of a first neural network layer; generate a key associated with the at least a portion of the first neural network layer; obtain the key from the storage device; obtain a version of the output data for input into a second neural network layer; and use the key to determine whether the version of the output data is different from the output data.

[0083] According to the implementation scheme, the storage device is a local storage device of the data processing system, and at least one processor is operable to: send output data to an external storage device outside the data processing system for storage; and obtain a version of the output data from the external storage device.

[0084] According to the implementation scheme, the storage device can be accessed by only one processor of the data processing system.

[0085] According to the implementation scheme, at least one processor is operable to: calculate a first signature based on a key and output data; store the first signature in a storage device; obtain the first signature from the storage device; and calculate a second signature based on a version of the key and output data, wherein, in order to determine whether the version of the output data is different from the output data, at least one processor is operable to determine whether the second signature is equal to the first signature.

[0086] According to the implementation scheme: an encryption engine that encrypts output data using a key; and a decryption engine that decrypts a version of the output data using a key to generate a decrypted version of the output data, wherein at least one processor is operable to determine whether a version of the output data is different from the output data based on the decrypted version of the output data.

[0087] According to the implementation scheme, in response to determining that the version of the output data is different from the output data, at least one processor is operable to generate data indicating that the version of the output data is different from the output data.

[0088] According to the implementation scheme, in response to determining that the version of the output data is the same as the output data, at least one processor is operable to process the version of the output data using a second neural network layer.

[0089] According to the implementation scheme, the data processing system is a first data processing system, the output data is a first output data, the key is a first key, and at least one processor is operable to: process a version of the output data using a second neural network layer to generate second output data; generate a second key, wherein the second key is associated with the second neural network layer; send the second output data to the second data processing system; and send the second key to the second data processing system.

[0090] According to the implementation plan, the encryption engine encrypts the second output data using a second key before sending the second output data to the second data processing system.

[0091] According to the implementation plan: the second key is sent to the second data processing system through the first data channel, which is different from the second data channel, and the second output data is sent to the second data processing system through the second data channel; or the second key is sent to the second data processing system using a shared storage device that can be accessed by both the first and second data processing systems.

[0092] According to the implementation scheme, the data processing system is a first data processing system, the key is a first key, and at least one processor is operable to: obtain a third key from a third data processing system, wherein the third key is associated with input data for input to at least a portion of the first neural network layer; obtain a version of the input data for input to at least a portion of the first neural network layer; and use the third key to determine whether the version of the input data is different from the input data.

[0093] According to the implementation scheme, the decryption engine uses a third key to decrypt a version of the input data to generate a decrypted version of the input data, wherein at least one processor is operable to determine whether a version of the input data is different from the input data based on the decrypted version of the input data.

[0094] According to the implementation scheme: in response to determining that the version of the input data is the same as the input data, at least one processor is operable to process the version of the input data using at least a portion of the first neural network layer; or in response to determining that the version of the input data is different from the input data, at least one processor is operable to generate data indicating that the version of the input data is different from the input data.

[0095] According to the implementation scheme, at least one processor is operable to determine, based on a location identifier, whether a portion of a version of the output data differs from a corresponding portion of the output data, wherein the location identifier identifies a corresponding portion of the output data within the output data.

[0096] According to the implementation scheme, at least one processor is operable to: modify a key based on a location identifier; and use the modified key to determine whether the version of that portion of the output data differs from the corresponding portion of the output data.

[0097] According to the implementation plan, the data processing system is an image processing system, the first neural network layer and the second neural network layer are corresponding layers of the convolutional neural network, the data input into the convolutional neural network represents the image, the output data is the output feature map, and the version of the output data is the input feature map.

[0098] According to the implementation scheme, at least one processor is operable to perform at least one of the following operations: applying lossless compression to output data; or applying lossy compression to output data, wherein a key is generated after lossy compression of the output data.

[0099] According to the present invention, a method includes: generating output data using at least a portion of a first neural network layer; generating a key associated with the at least a portion of the first neural network layer; storing the key in a storage device; obtaining the key from the storage device; obtaining a version of the output data for input into a second neural network layer; and using the key to determine whether the version of the output data is different from the output data.

[0100] In one aspect of the invention, the method includes at least one of the following: calculating a first signature based on a key and output data; storing the first signature in a storage device; obtaining the first signature from the storage device; and calculating a second signature based on a key and a version of the output data, wherein determining whether a version of the output data differs from the output data includes determining whether the second signature is equal to the first signature; or encrypting the output data using the key; and decrypting a version of the output data using the key, wherein determining whether a version of the output data differs from the output data is based on the decrypted version of the output data.

[0101] In one aspect of the invention, the method is implemented by a first data processing system, the key being a first key, and the method includes: obtaining a second key from a second data processing system, wherein the second key is associated with input data for input to at least a portion of a first neural network layer; obtaining a version of the input data for input to at least a portion of the first neural network layer; and using the second key to determine whether the version of the input data differs from the input data, wherein the second data processing system is a data processing system for an input device, wherein the input device includes at least one of: one or more sensors for acquiring sensor data and sending the sensor data to the first data processing system; and a decoder for outputting decoded data to the first data processing system.

Claims

1. A data processing system, the data processing system comprising: Storage device; and At least one processor, said at least one processor being used for: Output data is generated using at least a portion of the first neural network layer; Generate a key associated with at least a portion of the first neural network layer; Obtain the key from the storage device; Obtain a version of the output data used as input to the second neural network layer; as well as Using the key, determine whether the version of the output data is different from the output data itself. The at least one processor is operable to: Modify the key based on the location identifier; as well as Based on the location identifier, a modified key is used to determine whether a portion of the version of the output data differs from a corresponding portion of the output data, wherein the location identifier identifies the corresponding portion of the output data within the output data.

2. The data processing system according to claim 1, wherein the storage device is a local storage device of the data processing system, and the at least one processor is operable to: The output data is sent to an external storage device outside the data processing system for storage; and The version of the output data is obtained from the external storage device.

3. The data processing system of claim 1, wherein the storage device is accessible only by the at least one processor of the data processing system.

4. The data processing system according to claim 1, wherein the at least one processor is operable to: Calculate the first signature based on the key and the output data; The first signature is stored in the storage device; Obtain the first signature from the storage device; as well as A second signature is calculated based on the key and the version of the output data, wherein, in order to determine whether the version of the output data is different from the output data, the at least one processor is operable to determine whether the second signature is equal to the first signature.

5. The data processing system according to claim 1, wherein the data processing system comprises: An encryption engine that uses the key to encrypt the output data; and A decryption engine that uses the key to decrypt the version of the output data to generate a decrypted version of the output data, wherein the at least one processor is operable to determine, based on the decrypted version of the output data, whether the version of the output data is different from the output data.

6. The data processing system according to claim 1, wherein: In response to determining that the version of the output data is different from the output data, the at least one processor is operable to generate data indicating that the version of the output data is different from the output data; or In response to determining that the version of the output data is the same as the output data, the at least one processor is operable to process the version of the output data using the second neural network layer.

7. The data processing system of claim 1, wherein the data processing system is a first data processing system, the output data is first output data, the key is a first key, and in response to determining that the version of the output data is the same as the output data, the at least one processor is operable to: The second neural network layer is used to process the version of the output data to generate the second output data; Generate a second key, wherein the second key is associated with the second neural network layer; The second output data is sent to the second data processing system; as well as The second key is sent to the second data processing system, wherein: The second key is sent to the second data processing system through the first data channel, which is different from the second data channel. The second output data is sent to the second data processing system through the second data channel. or The second key is sent to the second data processing system using a shared storage device accessible to both the first and second data processing systems.

8. The data processing system of claim 1, wherein the data processing system is a first data processing system, the key is a first key, and the at least one processor is operable to: A third key is obtained from a third data processing system, wherein the third key is associated with input data used to input into at least a portion of the first neural network layer; Obtain a version of the input data for inputting into at least a portion of the first neural network layer; as well as Using the third key, determine whether the version of the input data is different from the input data, wherein: In response to determining that the version of the input data is the same as the input data, the at least one processor is operable to process the version of the input data using the at least a portion of the first neural network layer; or In response to determining that the version of the input data is different from the input data, the at least one processor is operable to generate data indicating that the version of the input data is different from the input data.

9. A method for data processing, the method comprising: Use at least a portion of the first neural network layer to generate the output data; Generate a key associated with at least a portion of the first neural network layer; The key is stored in a storage device; Obtain the key from the storage device; Obtain a version of the output data used as input to the second neural network layer; as well as, Using the key, determine whether the version of the output data is different from the output data itself. Determining whether the version of the output data is different from the output data includes: Modify the key based on the location identifier; as well as Based on the location identifier, a modified key is used to determine whether a portion of the version of the output data differs from a corresponding portion of the output data, wherein the location identifier identifies the corresponding portion of the output data within the output data.

Images