Apparatus for performing machine learning model inference
By combining trusted processing circuits and privacy circuits and utilizing random seed verification and encryption technology, the confidentiality protection problem of machine learning models during transmission and execution is solved, the security of models and user data is guaranteed, and the secure calculation and data protection of machine learning models are ensured.
Patent Information
- Application Number
- CN202410295066.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-03-14
- Publication Date
- 2025-09-16
AI Technical Summary
The confidentiality protection of machine learning models faces the risk of leakage during their provision and use, especially during data transmission and execution between model providers and users. Existing technologies make it difficult to effectively protect the confidentiality of model and user data.
Trusted processing circuits and privacy circuits are used to ensure the confidentiality of the model and the security of user data through random seed verification, encrypted machine learning models, blind signature input data and secure communication. The trusted neural network processing unit (TNPU) is used for secure computing to avoid data leakage.
It achieves confidentiality protection of machine learning models during transmission and execution, ensures the security of models and user data, prevents the leakage of confidential information, and improves the security of models and the protection level of user data.
Smart Images
Figure CN120654225A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to a device for executing a machine learning model, and more particularly to a device for executing a machine learning model using identity verification technology. Background Art
[0002] As the cost of training machine learning models continues to rise, protecting the confidentiality of machine learning models has become a critical issue in the sales and deployment of machine learning products. When model providers sell or provide machine learning models to users, they must only grant access to the model for their own use, ensuring that others cannot easily access the model's confidential information, including its structure and weights. Furthermore, the professional data used by enterprises to train or infer machine learning models is also highly confidential, and the risk of data leakage during model use must be avoided. This type of technology, which provides models to users while maintaining confidentiality, is called secure computing or confidential computing. Summary of the Invention
[0003] According to some embodiments of the present disclosure, a device for performing machine learning model inference is provided. The device includes a host and a trusted processing circuit. The host receives a random seed generated by an electronic device. The trusted processing circuit includes a privacy circuit. The privacy circuit has an embedded first public key and a first private key, and the privacy circuit generates a certificate based on the random seed, the first public key, and the first private key. The host outputs the certificate and the first public key to the electronic device for the electronic device to determine whether to output an encrypted machine learning model to the host. The privacy circuit decrypts the encrypted machine learning model to generate a machine learning model. The processing component array securely calculates the inference operation of the machine learning model.
[0004] In some embodiments, the first public key is registered in a database, so that the electronic device can query whether the trusted processing circuit is legitimate hardware based on the received first public key.
[0005] In some embodiments, the host includes a memory and a processor. The processor writes an encrypted machine learning model to the memory. The trusted processing circuit further includes a blind memory controller and a bus. The blind memory controller is coupled to the memory and reads the encrypted machine learning model from the memory. The bus transmits the encrypted machine learning model to the privacy circuit.
[0006] In some embodiments, the bus self-privacy circuit transmits the machine learning model to the oblivious memory controller, and the oblivious memory controller writes the machine learning model into the memory according to an access method different from that of the processor.
[0007] In some embodiments, the trusted processing circuit further includes a controller. During an inference operation, the controller instructs the obscure memory controller to read the first layer of the machine learning model. The processing component array generates a first feature based on the input data and the first layer. The controller determines whether to store the first feature in the memory or in a buffer memory of the trusted processing circuit based on the size of the first feature.
[0008] In some embodiments, the oblivious memory controller writes the prediction data generated by the inference operation to the memory according to the same access method as the processor.
[0009] According to some embodiments of the present disclosure, a device for performing machine learning model inference is provided, comprising a trusted processing circuit and a host. The trusted processing circuit includes a privacy circuit. The privacy circuit has a first public key and a first private key embedded therein. The host outputs the first public key to an electronic device, which checks whether the first public key is registered in a database to determine whether to transmit an encrypted machine learning model to the host. The privacy circuit decrypts the encrypted machine learning model using the first private key to generate a machine learning model. The trusted processing circuit securely computes inference operations on the machine learning model.
[0010] In some embodiments, the privacy circuitry decrypts the encrypted machine learning model to confirm a usage restriction status. When the usage restriction status includes a usage count, the trusted processing circuitry prohibits performing inference operations if the number of inference operations performed exceeds the usage count. When the usage restriction status includes a usage expiration date, the trusted processing circuitry prohibits performing inference operations if the current time is earlier than the usage expiration date.
[0011] In some embodiments, the privacy circuitry decrypts the encrypted machine learning model to confirm the usage restriction status. When the usage restriction status includes a restricted signature status, the host blinds the input data to generate blinded input data, and transmits the blinded input data to the electronic device to perform a blind signature operation on the input data. The host receives the second public key of the electronic device and a blinded signature generated by the electronic device based on the blinded input data, and unblinds the blinded signature to generate a signature. The trusted processing circuitry determines whether to perform an inference operation on the input data based on the second public key and the signature.
[0012] In some embodiments, a host includes a memory and a processor. The processor writes an encrypted machine learning model to the memory using a first addressing method. The trusted processing circuit further includes a memory controller. The memory controller writes the machine learning model to the memory using a second addressing method different from the first addressing method to prevent the processor from reading the machine learning model. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] To make the above and other objects, features, advantages and embodiments of the present disclosure more apparent and understandable, the accompanying drawings are described as follows:
[0014] Figure 1 is a schematic diagram of a system 10 according to some embodiments of the present disclosure;
[0015] Figure 2 According to some embodiments of the present disclosure, Figure 1 A flow chart of the method 20 of the system 10 is shown;
[0016] Figure 3 According to some embodiments of the present disclosure, Figure 2 A flowchart illustrating some steps of method 20; and
[0017] Figure 4 According to some embodiments of the present disclosure, Figure 2 A flow chart of some steps of method 20 is shown.
[0018] Component number description
[0019] 10 systems
[0020] 100 electronic devices
[0021] 110 trusted processing circuit
[0022] 111 controller
[0023] 112 bus
[0024] 113 controller
[0025] 114 buffer memory
[0026] 115 Privacy Circuit
[0027] 116 processing component array
[0028] 120 host
[0029] 121 processor
[0030] 122 memory
[0031] 200 electronic devices
[0032] 201 processor
[0033] 202 memory
[0034] 300 electronic devices
[0035] 301 processor
[0036] 302 memory
[0037] 303 database
[0038] 20 methods
[0039] Steps S10-S14
[0040] Steps S20-S24
[0041] Steps S30-S32
[0042] S40 steps
[0043] Steps S50-S56
[0044] S60 steps DETAILED DESCRIPTION
[0045] Various embodiments of the present disclosure are described below with reference to the accompanying drawings. In the drawings, like reference numerals represent like or similar components or method flows. Well-known components and steps are not described in the embodiments to avoid unnecessary limitations on the present disclosure.
[0046] Unless otherwise specified, the terms used in this disclosure generally have their ordinary meanings in the art. However, those skilled in the art will appreciate that the same component or procedure may be represented by different terms.
[0047] The terms "first," "second," and so on, as used herein, do not specifically refer to an order or sequence, nor are they intended to limit the present disclosure. They are simply used to distinguish components or operations described by the same technical terms. For example, a first component could be referred to as a second component, and similarly, a second component could be referred to as a first component. Such variations do not depart from the scope of the present disclosure.
[0048] Please refer to Figure 1 . Figure 1 FIG1 is a schematic diagram of a system 10 according to some embodiments of the present disclosure. The system 10 is used to perform secure computation of a machine learning model to ensure that the model confidentiality of the model provider and the data confidentiality of the model user are not leaked during the execution of the machine learning model.
[0049] For illustration, system 10 includes a model user's electronic device 100, a model provider's electronic device 200, and a hardware provider's electronic device 300. Electronic device 200 is used to store a machine learning model and provide the protected or encrypted machine learning model to electronic device 100. Electronic device 100 is used to store test data and perform inferences based on the machine learning model. Electronic device 300 is used to verify that the hardware in electronic device 100 has a qualified identity.
[0050] like Figure 1As shown, electronic device 100 includes a trusted processing circuit 110 and a host 120. Trusted processing circuit 100 is a processing circuit that supports trusted execution, such as a graphics processing unit (GPU) circuit that supports trusted execution. In some embodiments, trusted processing circuit 100 is a trusted neural processing unit (TNPU).
[0051] According to some embodiments, a trusted neural network processing unit (TNNPU) is a secure neural network accelerator that uses hardware to ensure that user data and model parameters remain confidential in an untrusted environment. Because the TNNPU can be customized to the neural network, it can achieve higher efficiency in secure neural network inference than some other methods.
[0052] In some embodiments, trusted processing circuit 110 includes a controller 101, a bus 102, a controller 103, a buffer 104, a privacy circuit 105, and a processing element array 106. In some embodiments, the bus is a system-on-a-chip (SoC) bus. Controller 103 is an oblivious random access memory (ORAM) controller. Buffer 104 is an on-chip buffer. Privacy circuit 105 includes privacy module circuitry that supports security-related functions.
[0053] In some embodiments, host 120 and electronic devices 200-300 are computer hosts, such as laptop computers, desktop computers, or mainframe computers. Host 120 and electronic devices 200 and 300 include processors 121, 201, and 301, respectively, and memories 122, 202, and 302, respectively. Electronic device 300 also includes a database 303.
[0054] According to some embodiments, the processors 121, 201, and 301 respectively include a central processing unit (CPU), or other programmable general-purpose or special-purpose microcontrol unit (MCU), microprocessor, digital signal processor (DSP), programmable controller, application specific integrated circuit (ASIC), graphics processing unit (GPU), arithmetic logic unit (ALU), complex programmable logic device (CPLD), field programmable gate array (FPGA), or other similar components or combinations of the above components.
[0055] The memories 122 , 202 , and 302 respectively include static random access memory (SRAM), dynamic random access memory (DRAM), other suitable memories, or a combination thereof.
[0056] exist Figure 1 In the illustrated embodiment, the trusted processing circuit 110 is coupled to a host 120. The host 120 is coupled to an electronic device 200. The electronic device 200 is coupled to an electronic device 300. It should be understood that, in the embodiments, the term "coupled" may refer to either an indirect electrical coupling of one component to another component through another component, or a direct electrical coupling of one component to another component without requiring another component.
[0057] For example, in some embodiments, the trusted processing circuit 110 is directly coupled to an input / output (I / O) interface of the host 120. The host 120 is connected to the electronic device 200 via the Internet. The electronic device 200 is connected to the electronic device 300 via the Internet.
[0058] In some embodiments, bus 102 couples controllers 101 and 103, buffer memory 104, privacy circuit 105, and processing element array 106. Controller 101 couples processor 201. Controller 103 couples memory 202. Processor 201 couples memory 202. Processor 301 couples memory 302. Processor 401 couples memory 402 and database 403.
[0059] According to some embodiments, when the trusted processing circuit 110 leaves the factory, the identity (ID) and public key pkt of the trusted processing circuit 110 are registered with the electronic device 300. In some embodiments, the database 303 is used to store the identities and public keys of all trusted processing circuits manufactured by the hardware provider, including the ID and public key pkt of the trusted processing circuit 110. In some embodiments, the database 303 is an unalterable database, such as a blockchain database or a database secured by any suitable tool.
[0060] Processor 301 is used to perform database 303 query operations and secure communication between electronic devices 300 and 200. Secure communication ensures confidentiality of data transmitted between the two devices through encryption, for example. Memory 302 is used to perform data temporary storage operations in electronic device 300.
[0061] The processor 201 is used to perform secure communications with the trusted processing circuit 110, the host 121, and the electronic device 300. The processor 201 is also used to encrypt a machine learning model stored in the electronic device 200. The processor 201 is also used to execute an identification scheme for the electronic device 100. The processor 201 is also used to perform blind signature operations on data from the electronic device 100. The memory 202 is used to temporarily store data in the electronic device 200.
[0062] The processor 121 is configured to perform secure communication with the electronic device 200 and the trusted processing circuit 110. The processor 201 is also configured to transmit data to the trusted processing circuit 110 to perform secure computations on encrypted machine learning models. The processor 201 is also configured to request the electronic device 200 to perform blind signatures.
[0063] In some embodiments, the controller 111 is configured to receive instructions from the processor 121. In some embodiments, the controller 111 and the privacy circuit 115 are configured to perform secure communications with the electronic device 200 and the host 120, respectively.
[0064] The bus 112 is used to transmit data and signals between the controller 111 , the controller 113 , the buffer memory 114 , the privacy circuit 115 , and the processing element array 116 .
[0065] The controller 113 is used to read and write to the memory 122. The controller 113 is used to protect data access between the trusted processing circuit 110 and the memory 122, ensuring that the host 120 cannot obtain confidential information in the data written by the controller 113 to the memory 122. Specifically, in some embodiments, the controller 113 uses different access patterns than the processor 121 to protect confidential information in the data written by the controller 113 to the memory 122.
[0066] The buffer memory 114 is used to perform temporary storage operations during internal calculations of the trusted processing circuit 110 .
[0067] In some embodiments, the trusted processing circuit 110 is used to store its own ID, public key pkt, and private key skt. Devices or circuits outside the trusted processing circuit 110 (e.g., the host 120) cannot access the private key skt, and therefore, the model user cannot obtain the private key skt. In some embodiments, one or more of the ID, public key pkt, and private key skt of the trusted processing circuit 110 is embedded in the privacy circuit 115. For example, in some embodiments, one or more of the ID, public key pkt, and private key skt of the trusted processing circuit 110 is a password of the physical unclonable function (PUF) of the privacy circuit 115. For example, the privacy circuit 115 includes a one-time programmable (OTP) memory that stores one or more of the ID, public key pkt, and private key skt of the trusted processing circuit 110.
[0068] The processing element array 116 is used to perform inference acceleration operations of the machine learning model.
[0069] Figure 1 The configuration is given for illustrative purposes. Figure 1 Various embodiments are within the contemplated scope of this disclosure. For example, Figure 1 The coupling shown can be replaced by signal connection methods such as wireless transmission, optical transmission, etc. For example, the electronic device 200 is connected to the electronic device 300 via wireless communication. The electronic device 300 is connected to the electronic device 400 via wireless communication.
[0070] Please refer to Figures 2 to 4 . Figure 2 According to some embodiments of the present disclosure, Figure 1 A flow chart of a method 20 of the system 10 is shown. Figure 3 and Figure 4 According to some embodiments of the present disclosure, Figure 2 A flow chart of some steps of method 20 is shown.
[0071] like Figure 2 As shown, method 20 includes steps S10-S60. In some embodiments, as Figure 3 As shown, step S10 includes steps S11-S14 and step S20 includes steps S21-S24. In some embodiments, as Figure 4 As shown, step S30 includes steps S31-S32 and step S50 includes steps S51-S56.
[0072] In some embodiments, method 20 operates the operating system 10 to perform secure machine learning model inference, ensuring the confidentiality of data transmission and machine learning model inference through identity verification methods, encrypted machine learning models, blind signature input data, secure communication and secure execution.
[0073] Please refer to Figures 1 to 3 Steps S10-S20 execute the identity verification method of the trusted processing circuit 110. In step S10, the system 10 generates an identity proof of the trusted processing circuit 110. In step S20, the system 10 verifies the identity proof of the trusted processing circuit 110.
[0074] like Figure 3 As shown, in step S11, the processor 201 executes a random seed generation operation RandomSeed() to generate a random seed R.
[0075] In step S12 , the processor 201 performs secure communication with the trusted processing circuit 110 to output the random seed R to the trusted processing circuit 110 . In some embodiments, the processor 201 outputs the random seed R to the host 120 , and the host 120 outputs the random seed R to the trusted processing circuit 110 .
[0076] In step S13, the electronic device 100 generates a certificate P proving the existence of a legitimate trusted processing circuit 110. In some embodiments, the trusted processing circuit 110 performs a commit operation Commit(R, pkt, skt) of the authentication method based on the random seed R, the public key pkt, and the private key skt to generate the certificate P. In some embodiments, the controller 111 receives an instruction from the processor 121 to request the privacy circuit 115 to perform the commit operation.
[0077] In step S14 , the trusted processing circuit 110 performs secure communication to output the certificate P, the public key pkt, and the ID to the host 120 . The host 120 then outputs the certificate P, the public key pkt, and the ID to the electronic device 200 .
[0078] In step S21, the processor 201 performs a verify operation (Verify(pkt, P, R)) of the identity authentication method based on the public key pkt, the certificate P, and the random seed R to generate a verification result V. Based on the verification result V, the processor 201 determines whether the trusted processing circuit 110 possesses the private key skt that matches the public key pkt. For example, when the verification result V has a first value, such as True, the processor 201 determines that the trusted processing circuit 110 possesses the private key skt corresponding to the public key pkt. Conversely, when the verification result V has a second value different from the first value, such as False, the processor 201 determines that the trusted processing circuit 110 does not possess the private key skt corresponding to the public key pkt.
[0079] In step S22, the processor 201 performs secure communication to output the public key pkt and the ID to the electronic device 300. In some embodiments, when the processor 201 determines that the trusted processing circuit 110 has the private key skt corresponding to the public key pkt, the processor 201 outputs the public key pkt and the ID to the electronic device 300. When the processor 201 determines that the trusted processing circuit 110 does not have the private key skt corresponding to the public key pkt, the processor 201 does not output the public key pkt and the ID to the electronic device 300.
[0080] In step S23, processor 301 performs a check operation Check(pkt, ID) based on the public key pkt and ID to generate a check result C. In the check operation Check(pkt, ID), processor 301 queries whether the public key pkt and ID from processor 201 are registered (i.e., whether they are recorded in the same registration data in database 303). If the public key pkt and ID are registered, it indicates that the trusted processing circuit with the public key pkt and ID has been shipped. If processor 301 determines that the public key pkt and ID are registered, processor 301 generates a check result C having a first value (e.g., True). If processor 301 determines that the public key pkt and ID are not registered, processor 301 generates a check result C having a second value (e.g., False) different from the first value.
[0081] In step S24, the processor 301 performs secure communication and outputs the check result C to the electronic device 200. In some embodiments, the processor 201 determines whether the public key pkt and the ID are registered based on the check result C. When the check result C has a first value (e.g., True), the processor 201 determines that the public key pkt and the ID are registered. Conversely, when the check result C has a second value different from the first value (e.g., False), the processor 201 determines that the public key pkt and the ID are not registered.
[0082] In some embodiments, the processor 201 determines whether the electronic device 100 has passed verification (i.e., whether it has a legitimate trusted processing circuit 110) based on the verification result V and the check result C. When the verification result V and the check result C have a first value (e.g., True), the processor 201 determines that the electronic device 100 has passed verification. When the verification result V and / or the check result C have a second value (e.g., False), the processor 201 determines that the electronic device 100 has failed verification. In some embodiments, passing verification for the electronic device 100 indicates that only the trusted processing circuit 110 has the private key skt that can decrypt the public key pkt.
[0083] Please refer to Figures 1 to 4 In step S30, the system 10 encrypts the machine learning model. In some embodiments, when the electronic device 100 is determined to have passed the verification in step 20, the system 10 executes step S30. Conversely, when the electronic device 100 is determined to have failed the verification in step 20, the system 10 does not execute step S30.
[0084] In step S31 , the processor 201 performs an encryption operation Enc(pkt, M) according to the public key pkt to encrypt the machine learning model M stored in the electronic device 200 to generate an encrypted machine learning model M′.
[0085] In some embodiments, the model provider determines usage restrictions of the machine learning model M. During the encryption process, the processor 201 stores a restriction state indicating the usage restrictions of the machine learning model M in the encrypted machine learning model M′.
[0086] In some embodiments, the model provider determines whether the input data (test data) requires a signature to restrict the input data on which the model user can perform inference. When the model provider determines that the input data (test data) requires a signature, the processor 201 stores a restriction state including a state s in the encrypted machine learning model M' in step S31, where the state s indicates that the input data (test data) requires a signature.
[0087] In some embodiments, the model provider determines the number of times the machine learning model M can be used to limit the number of inferences that the model user can perform. When the model provider determines that the machine learning model M has a usage limit, the processor 201 stores a restriction state including a state c in the encrypted machine learning model M' in step S31, where the state c indicates that the machine learning model M has a usage limit. In some embodiments, the state c includes the number of times N that the machine learning model M is limited in usage.
[0088] In some embodiments, the model provider determines the usage time of the machine learning model M to limit the period during which the model user can perform inferences. When the model provider determines that the machine learning model M has a usage time limit, the processor 201 stores a restriction state including a state t in the encrypted machine learning model M' in step S31, where the state t indicates that the machine learning model M has a usage time limit. In some embodiments, the state t includes the usage time T during which the machine learning model M is restricted. In some embodiments, the usage time T is the usage period of the machine learning model M.
[0089] In some embodiments, when the model provider determines that the machine learning model M has no usage restrictions, the processor 201 stores a restriction state including state n in the encrypted machine learning model M' in step S31, where state n indicates that the machine learning model M has no usage restrictions.
[0090] In step S32, the processor 201 performs secure communication with the trusted processing circuit 110 to output the encrypted machine learning model M' to the trusted processing circuit 110. In some embodiments, the processor 121 receives the encrypted machine learning model M' and outputs the encrypted machine learning model M' to the controller 111.
[0091] In step S40, the system 10 determines whether the input data (test data) of the machine learning model M requires a signature. In some embodiments, if the processor 121 determines that the restricted state includes state s (i.e., the machine learning model M only accepts signed input data), the system 10 executes step S50. Conversely, if the processor 121 determines that the restricted state does not include state s, the system 10 does not execute step S50.
[0092] In step S50 , the system 10 blindly signs the input data (test data) X stored in the electronic device 100 to generate signed test data that is allowed to be used by the machine learning model M. The input data X is the test data that the model user wants to use the machine learning model M to infer.
[0093] In step S51 , the processor 121 performs a blinding operation Blinding(X) in blind signature on the input data X to generate blinded data X′.
[0094] In step S52 , the processor 121 performs secure communication with the electronic device 200 to output the blinded data X′ to the electronic device 200 .
[0095] In step S53, the processor 201 executes a blind signature key generation operation GenSignKey to generate a private key sks and a public key pks. The system 10 performs a blind signature based on the private key sks and the public key pks.
[0096] In step S54 , the processor 201 performs a sign operation Sign(sks, X′) in a blind signature on the blinded data X′ according to the private key sks to generate a blinded signature S′.
[0097] In step S55 , the processor 201 performs secure communication with the host 120 to output the blinded signature S′ to the host 120 .
[0098] In some embodiments, the processor 201 further performs secure communication with the trusted processing circuit 110 to output the public key pks to the trusted processing circuit 110 .
[0099] In step S56 , the processor 121 performs an unblinding operation UnBlinding(S′) on the blinded signature S′ to generate a signature S.
[0100] like Figure 4 As shown, when the system 10 determines in step S40 that the input data requires a signature, the system 10 executes step S50 and then executes step S60. Conversely, when the system 10 determines in step S40 that the input data does not require a signature, the system 10 directly executes step S60.
[0101] In step S60 , the system 10 securely executes inference of the machine learning model M. In some embodiments, the controller 103 is configured to perform oblivious read and write operations (eg, using a different addressing scheme than the processor 121 ) to achieve secure execution of inference.
[0102] In some embodiments, the processor 121 stores the encrypted machine learning model M′ into the memory 122 in step S60.
[0103] The controller 111 is configured to request the privacy circuit 115 to decrypt the encrypted machine learning model M′ into the machine learning model M. The controller 113 stores the machine learning model M in the memory 122. In some embodiments, the controller 113 writes the machine learning model M into the memory 122 without any change.
[0104] In some embodiments, the processor 121 stores the input data X in the memory 122 , and the input data X serves as input data for the controller 111 to perform inference operations of the machine learning model M.
[0105] The controller 111 decrypts the machine learning model M′ to obtain the restriction state of the machine learning model M, and the controller 111 confirms the state included in the restriction state of the machine learning model M.
[0106] When the controller 111 confirms that the restricted state of the machine learning model M includes state s (ie, the input data needs to be signed), the processor 121 stores the signature S and the public key pks in the memory 122 .
[0107] Next, the controller 111 performs a verification operation Verify(pks, X, S) in the blind signature to determine whether the signature S is correct (determine whether the input data X matches the signature S) based on the public key pks and the input data X. In some embodiments, the controller 111 performs the verification operation Verify(pks, X, S) based on the public key pks, the input data X, and the signature S to generate a verification result. When the verification result has a first value (e.g., True), the controller 111 determines that the signature S is correct. Conversely, when the verification result has a second value different from the first value (e.g., False), the controller 111 determines that the signature S is incorrect.
[0108] When the controller 111 determines that the signature S is correct, the controller 111 then performs a model inference operation Model(M,X) of the machine learning model M on the input data X to generate prediction data Y.
[0109] When the controller 111 determines that the restricted state of the machine learning model M includes state c (i.e., there is a usage limit), the controller 111 stores a count value in the buffer memory 114. In some embodiments, the initial value of the count value is a value of "0" or "1." Each time the controller 111 executes the model inference operation Model(M, X), the controller 111 increments the count value by a value of "1." When the count value exceeds the restricted number of uses N, the controller 111 no longer executes (prohibits) the model inference operation Model(M, X).
[0110] When the controller 111 confirms that the restricted state of the machine learning model M includes state t (i.e., there is a usage time limit), the controller 111 confirms whether the current time is within the restricted usage time T before each execution of the model inference operation Model(M,X). When the controller 111 confirms that the current time is earlier than or equal to the restricted usage time T, the controller 111 executes the model inference operation Model(M,X). When the controller 111 confirms that the current time is later than the restricted usage time T, the controller 111 no longer executes the model inference operation Model(M,X).
[0111] When the controller 111 confirms that the restricted state of the machine learning model M includes state n (ie, there are no restrictions), the controller 111 directly executes the model inference operation Model(M,X).
[0112] According to some embodiments, in the model inference operation Model(M, X), the controller 111 requests the controller 113 to read the first layer (e.g., including the weights of the first layer) of the machine learning model M from the memory 122. The processing component array 116 generates the features of the first layer based on the input data X and the first layer inference of the machine learning model M. The controller 111 determines whether to store the features of the first layer in the buffer memory 114 or the memory 122 based on the size (occupied capacity) of the features of the first layer. When the size of the features of the first layer is less than a threshold, the features of the first layer are stored in the buffer memory 114. Conversely, when the size of the features of the first layer is greater than the threshold, the features of the first layer are stored in the memory 122.
[0113] Next, controller 111 instructs controller 113 to read the second layer of machine learning model M from memory 122. Processing component array 116 generates second-layer features based on the first-layer features of machine learning model M and the second-layer inferences of machine learning model M. Similar to the operation described above, controller 111 determines whether to store the second-layer features in buffer memory 114 or memory 122 based on the size of the second-layer features.
[0114] The controller 111 repeatedly performs operations similar to the above for the remaining layers of the machine learning model M to generate features and prediction data Y for each layer.
[0115] Controller 111 requests controller 113 to store the predicted data Y in memory 122 in an unencrypted form (e.g., without using a blind write operation). When controller 111 completes the model inference operation Model(M, X), controller 111 generates an end signal to processor 121, indicating that the model inference operation Model(M, X) has ended.
[0116] It should be understood that Figures 2 to 4The steps of the method 20 shown in FIG. 20 may be adjusted in sequence according to actual needs, or may even be performed simultaneously or partially simultaneously, unless the sequence is specifically stated. Figures 2 to 4 Additional operations are provided before, during, and after the steps shown, and some of the operations described above may be replaced or eliminated to obtain additional embodiments of method 20.
[0117] In summary, the present disclosure provides a system and method for performing inference on a machine learning model. The system encrypts a model provider's machine learning model to protect confidential information within the model. Furthermore, the system authenticates the trusted processing circuitry of the model user, ensuring that only the trusted processing circuitry can decrypt the encrypted machine learning model. The system also supports secure communication and secure computation to prevent the disclosure of confidential information within the model.
[0118] The features of several embodiments are summarized above so that those skilled in the art can better understand the various aspects of an embodiment of the present invention. Those skilled in the art should understand that they can easily use an embodiment of the present invention as a basis for designing or modifying other processes and structures to achieve the same purposes and / or achieve the same advantages of the embodiments introduced herein. Those skilled in the art should also recognize that these equivalent structures do not depart from the spirit and scope of an embodiment of the present invention, and these equivalent structures can be variously modified, replaced, and changed herein without departing from the spirit and scope of an embodiment of the present invention.
Claims
1. A device for performing inference on a machine learning model, characterized in that: Include: a host, configured to receive a random seed generated by an electronic device; and A trusted processing circuit comprising: a privacy circuit having a first public key and a first private key embedded therein, the privacy circuit being configured to generate a certificate based on the random seed, the first public key, and the first private key; The host is further configured to output the certificate and the first public key to the electronic device so that the electronic device can determine whether to output an encrypted machine learning model to the host. wherein the privacy circuit is further configured to decrypt the encrypted machine learning model to generate a machine learning model; and An array of processing components for securely computing an inference operation of the machine learning model.
2. The device according to claim 1, characterized in that The first public key is registered in a database, so that the electronic device can query whether the trusted processing circuit is legitimate hardware based on the received first public key.
3. The device according to claim 1, characterized in that The host comprises: a memory; and a processor, configured to write the encrypted machine learning model into the memory, Wherein the trusted processing circuit further comprises: a blind memory controller coupled to the memory and configured to read the encrypted machine learning model from the memory; and A bus is used to transmit the encrypted machine learning model to the privacy circuit.
4. The device according to claim 3, characterized in that The bus is further used to transmit the machine learning model from the privacy circuit to the obscure memory controller, The obscure memory controller is further configured to write the machine learning model into the memory according to an access method different from that of the processor.
5. The device according to claim 4, characterized in that The trusted processing circuit further comprises: a controller for requesting the obscure memory controller to read a first layer of the machine learning model during the inference operation; The processing component array is further configured to generate a first feature based on input data and the first layer, and the controller is further configured to determine whether to store the first feature in the memory or a buffer memory of the trusted processing circuit based on the size of the first feature.
6. The device according to claim 4, characterized in that The obscure memory controller is further configured to write prediction data generated by the inference operation into the memory according to an access method identical to that of the processor.
7. A device for performing inference on a machine learning model, characterized in that Include: A trusted processing circuit comprising: a privacy circuit having a first public key and a first private key embedded therein; and a host, configured to output the first public key to an electronic device, for the electronic device to check whether the first public key is registered in a database to determine whether to transmit an encrypted machine learning model to the host, wherein the privacy circuit is further configured to decrypt the encrypted machine learning model according to the first private key to generate a machine learning model; and The trusted processing circuit is used to securely compute an inference operation of the machine learning model.
8. The device according to claim 7, characterized in that The privacy circuit is further configured to decrypt the encrypted machine learning model to confirm a usage restriction status; wherein when the usage restriction status includes a usage count, the trusted processing circuit prohibits execution of the inference operation when the number of times the inference operation is executed is greater than the usage count; and When the usage restriction status includes a usage period, the trusted processing circuit prohibits executing the inference operation when the current time is earlier than the usage period.
9. The device according to claim 7, characterized in that The privacy circuit is further configured to decrypt the encrypted machine learning model to confirm a usage restriction status; When the usage restriction status includes a restricted signature status, the host is further configured to blind an input data to generate blinded input data, and transmit the blinded input data to the electronic device to perform a blind signature operation on the input data; The host is further configured to receive a second public key of the electronic device and a blinded signature generated by the electronic device according to the blinded input data, and to unblind the blinded signature to generate a signature; as well as The trusted processing circuit determines whether to perform the inference operation on the input data according to the second public key and the signature.
10. The device according to claim 7, characterized in that The host comprises: a memory; and a processor configured to write the encrypted machine learning model into the memory according to a first addressing method, wherein the trusted processing circuit further comprises: A memory controller is used to write the machine learning model into the memory according to a second addressing method different from the first addressing method to prevent the processor from reading the machine learning model.