End side model calling system and method and computer readable storage medium

Abstract

The invention discloses an end side model calling system and method and a computer readable storage medium. On the basis of an end-side model calling system, only when a main core receives an authorization notification, decryption operation is carried out on encrypted end-side model parameters on the basis of an encryption engine in an NPU, that is, the end-side model parameters exist in an encrypted form in a general state, and the security of the end-side model parameters can be ensured; decryption operation is provided by the encryption engine, so that the response time can be shortened; in addition, based on the SMAC in the firewall and the NPU, a high-bandwidth and high-capacity secure memory can be provided for the operation of the neural network model, so that the operation efficiency of the neural network model is greater than the preset efficiency. According to the embodiment of the invention, both the end-side model parameter safety and the end-side model reasoning operation performance are considered.

Description

Technical Field

[0001] The present application relates to the field of data security technology, and in particular to a terminal-side model calling system, method, and computer-readable storage medium. Background Art

[0002] The end-side model refers to a neural network model that runs locally on a device (such as a smartphone, cockpit device, IoT device, embedded system, etc.). Since the end-side model stores model parameters, processes data, and performs inference operations locally, the model parameters and the execution environment of the inference operations of the end-side model face the potential risk of being attacked. However, common traditional security mechanisms, such as encrypted storage of model parameters, have strengthened the protection of model parameters to a certain extent, but have also reduced the performance of the end-side model inference operations. Summary of the invention

[0003] The present application provides an end-side model calling system, method and computer-readable storage medium, which can solve the technical problem in the prior art that model parameter security and end-side model reasoning operation performance cannot be taken into account at the same time.

[0004] In a first aspect, an embodiment of the present application provides an end-side model calling system, wherein the end-side model calling system includes a firewall and a neural network processor NPU, wherein the NPU internally integrates a master core, a slave core, an encryption engine, and a secure memory access controller SMAC, wherein: The main core is used to obtain the encrypted terminal model parameters when receiving the authorization notification; An encryption engine, used to decrypt the encrypted end-side model parameters to obtain the end-side model parameters, where the end-side model parameters are used to configure a neural network model running in the slave core; The firewall and SMAC are used to allocate secure memory for the neural network model operation so that the operation efficiency of the neural network model is greater than the preset efficiency.

[0005] In combination with the first aspect, in one embodiment, the end-side model calling system also includes a trusted application running in a trusted execution environment TEE, which is used to verify the legitimacy of the signature data when it is received from the algorithm authorization server; if the legitimacy verification passes, an authorization notification is sent to the main core.

[0006] In combination with the first aspect, in one implementation, the end-side model calling system further includes a cloud server, an algorithm authorization server, and a client application running in a rich execution environment REE, wherein: The client application is used to send a terminal model call request to the trusted application; The trusted application is used to send a signed random number to the client application when receiving a terminal model call request sent by the client application; The client application is used to send the received signature random number to the cloud server; The cloud server is used to detect whether the received signature random number is legal. If it is legal, it sends an authorization request to the algorithm authorization server; The algorithm authorization server is used to sign the authorization field to obtain signature data when receiving the authorization request, and send the signature data to the cloud server; The cloud server is used to send the received signature data to the client application; The client application sends the received signed data to the trusted application.

[0007] In combination with the first aspect, in one implementation, the algorithm authorization server is further used to obtain identification information of the client application, and generate and save an authorization record based on the time when the signature data is sent and the identification information.

[0008] In a second aspect, an embodiment of the present application provides a terminal-side model calling method, and the terminal-side model calling method includes: When receiving the authorization notification, the main core obtains the encrypted terminal-side model parameters; The encryption engine decrypts the encrypted end-side model parameters to obtain the end-side model parameters, which are used to configure the neural network model running in the slave core. Secure memory is allocated to the neural network model operation through the firewall and SMAC so that the operation efficiency of the neural network model is greater than the preset efficiency.

[0009] In conjunction with the second aspect, in an implementation manner, before the main core obtains the encrypted terminal-side model parameters upon receiving the authorization notification, the method further includes: When a trusted application running in the trusted execution environment TEE receives signature data sent by the algorithm authorization server, it verifies the legitimacy of the signature data; If the legitimacy verification passes, an authorization notification is sent to the main core.

[0010] In conjunction with the second aspect, in one implementation, when the trusted application running in the trusted execution environment TEE receives the signature data issued by the algorithm authorization server, before the legitimacy verification of the signature data is performed, it also includes: The client application running in the rich execution environment REE sends a terminal model call request to the trusted application; When the trusted application receives the end-side model call request sent by the client application, it sends a signed random number to the client application; The client application sends the received signature random number to the cloud server; The cloud server checks whether the received signature random number is legal. If it is legal, it sends an authorization request to the algorithm authorization server; When the algorithm authorization server receives the authorization request, it signs the authorization field to obtain the signature data and sends the signature data to the cloud server; The cloud server sends the received signature data to the client application; The client application sends the received signed data to the trusted application.

[0011] In conjunction with the second aspect, in one implementation, the terminal-side model calling method further includes: The algorithm authorization server obtains the identification information of the client application, and generates and saves the authorization record based on the time when the signature data is sent and the identification information.

[0012] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, on which an end-side model calling program is stored. When the end-side model calling program is executed by a processor, the steps of the end-side model calling method described in the second aspect are implemented.

[0013] The beneficial effects brought by the technical solution provided by the embodiments of the present application include: In the embodiment of the present application, only when the main core receives the authorization notification, the encrypted end-side model parameters are decrypted based on the encryption engine inside the NPU (Neural network Processing Unit), that is, under normal conditions, the end-side model parameters exist in encrypted form, which can ensure the security of the end-side model parameters; the decryption operation is provided by the encryption engine, which can shorten the response time; in addition, based on the firewall and the SMAC (Secure Memory Access Control) inside the NPU, high-bandwidth and high-capacity secure memory can be provided for the neural network model operation, so that the operation efficiency of the neural network model is greater than the preset efficiency. That is, through the embodiment of the present application, both the security of the end-side model parameters and the performance of the end-side model reasoning operation are taken into account. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 This is a schematic diagram of the architecture of an embodiment of a terminal-side model calling system of the present application; Figure 2 This is a schematic diagram of the architecture of another embodiment of the terminal-side model calling system of the present application; Figure 3 This is a flow chart of an embodiment of a terminal-side model calling method of the present application. DETAILED DESCRIPTION

[0015] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0016] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0017] In a first aspect, an embodiment of the present application provides a terminal-side model calling method.

[0018] In one embodiment, referring to Figure 1 , Figure 1 This is a schematic diagram of the architecture of an embodiment of the terminal model calling system of the present application. Figure 1 As shown, the end-side model calling system includes a firewall and a neural network processor NPU, and the NPU integrates a master core, a slave core, an encryption engine, and a secure memory access controller SMAC, wherein: The main core is used to obtain the encrypted terminal model parameters when receiving the authorization notification; An encryption engine, used to decrypt the encrypted end-side model parameters to obtain the end-side model parameters, where the end-side model parameters are used to configure a neural network model running in the slave core; The firewall and SMAC are used to allocate secure memory for the neural network model operation so that the operation efficiency of the neural network model is greater than the preset efficiency.

[0019] In the embodiment of the present application, the end-side model can be a neural network model deployed locally on the device, and the end-side model parameters are obtained based on the training process. This part is a conventional technology and will not be described here. Among them, the neural network model can be a deep neural network model DNN or a recurrent neural network model RNN, and the neural network model runs in the slave core.

[0020] In order to ensure the security of the end-side model parameters, the end-side model parameters are generally stored in an encrypted form, for example, the encrypted end-side model parameters are stored in the trusted execution environment TEE. When the main core receives the authorization notification, it means that the end-side model is authorized for use, and then obtains the encrypted end-side model parameters, and decrypts the encrypted end-side model parameters through the encryption engine to obtain the end-side model parameters.

[0021] After obtaining the end-side model parameters, they can be configured in the neural network model. The configuration action can be completed by the master core or the slave core.

[0022] In order to ensure that the operation efficiency of the neural network model is greater than the preset efficiency, it is necessary to allocate sufficient secure memory for the neural network model operation, which is achieved by the firewall and SMAC. Specifically, the firewall sets the size of the secure memory (for example, 2G), and sets the slave core as an object that can access the secure memory; SMAC sets the size of the secure memory that the slave core can access (for example, 500M). It should be noted that the numerical values ​​here are only for example purposes and do not constitute a limitation on the embodiments of the present application. The numerical settings are based on actual needs.

[0023] In the embodiment of the present application, only when the main core receives the authorization notification, the encrypted end-side model parameters are decrypted based on the encryption engine inside the NPU (Neural network Processing Unit), that is, under normal conditions, the end-side model parameters exist in encrypted form, which can ensure the security of the end-side model parameters; the decryption operation is provided by the encryption engine, which can shorten the response time; in addition, based on the firewall and the SMAC (Secure Memory Access Control) inside the NPU, high-bandwidth and high-capacity secure memory can be provided for the neural network model operation, so that the operation efficiency of the neural network model is greater than the preset efficiency. That is, through the embodiment of the present application, both the security of the end-side model parameters and the performance of the end-side model reasoning operation are taken into account.

[0024] Furthermore, in one embodiment, the end-side model calling system also includes a trusted application running in a trusted execution environment TEE, which is used to verify the legitimacy of the signature data when it is received from the algorithm authorization server; if the legitimacy verification passes, an authorization notification is sent to the main core.

[0025] In this embodiment, the use of the end-side model needs to be authorized by the algorithm authorization server. In order to avoid the occurrence of forged authorization, the signature data issued by the algorithm authorization server is verified for legitimacy by a trusted application running in the trusted execution environment TEE. Only when the legitimacy verification passes, the authorization notification is sent to the main core.

[0026] Furthermore, in one embodiment, the end-side model calling system further includes a cloud server, an algorithm authorization server, and a client application running in a rich execution environment REE, wherein: The client application is used to send a terminal model call request to the trusted application; The trusted application is used to send a signed random number to the client application when receiving a terminal model call request sent by the client application; The client application is used to send the received signature random number to the cloud server; The cloud server is used to detect whether the received signature random number is legal. If it is legal, it sends an authorization request to the algorithm authorization server; The algorithm authorization server is used to sign the authorization field to obtain signature data when receiving the authorization request, and send the signature data to the cloud server; The cloud server is used to send the received signature data to the client application; The client application sends the received signed data to the trusted application.

[0027] In this embodiment, refer to Figure 2 , Figure 2 This is a schematic diagram of the architecture of another embodiment of the terminal model calling system of the present application. Figure 2 As shown, the client application running in the rich execution environment REE sends an end-side model call request to the trusted application; when the trusted application receives the end-side model call request sent by the client application, it generates a random number through a random number generator, then signs it with a local private key, and sends the signed random number to the client application; the client application forwards the signed random number to the cloud server; the cloud server uses the client's certificate to detect whether the received signed random number is legal, and detects whether the signed random number has been received before. If it has not been received before, it means that the received signed random number is legal, and then sends an authorization request to the algorithm authorization server. Correspondingly, if the signed random number If it has been received before, it means that the received signature random number is illegal, and the authorization request will not be sent to the algorithm authorization server. Based on this processing, replay attacks can be prevented; when the algorithm authorization server receives the authorization request, it signs the authorization field to obtain signature data, and sends the signature data to the cloud server, where the authorization field can be a pre-defined field, and the signature data can be an encrypted field obtained by encrypting the authorization field with a private key, or the encrypted field and the authorization field itself; the cloud server forwards the signature data to the client application; the client application forwards the signature data to the trusted application. At this point, the trusted application can verify the legitimacy of the signature data, which is the digital signature verification process.

[0028] Furthermore, in one embodiment, the algorithm authorization server is also used to obtain identification information of the client application, and generate and save an authorization record based on the time when the signature data is sent and the identification information.

[0029] In this embodiment, the algorithm authorization server may actively obtain the identification information of the client application, or the identification information of the client application may be attached during the transmission of the signature random number. The identification information of the client application may be the user account used to log in to the client application.

[0030] An authorization record is generated and saved based on the time when the signature data is sent and the identification information, which is equivalent to saving a record of the client application using the end-side model. Based on this record, billing information can be further generated to realize paid use of the end-side model.

[0031] In a second aspect, an embodiment of the present application also provides a terminal-side model calling device.

[0032] In one embodiment, referring to Figure 3 , Figure 3 This is a flow chart of an embodiment of the terminal model calling method of the present application. Figure 3 As shown, the terminal-side model calling method includes: Step S10: When receiving the authorization notification, the main core obtains the encrypted terminal-side model parameters; Step S20: the encryption engine decrypts the encrypted terminal model parameters to obtain the terminal model parameters, where the terminal model parameters are used to configure the neural network model running in the slave core. Step S30, allocating secure memory for neural network model calculations through the firewall and SMAC, so that the calculation efficiency of the neural network model is greater than the preset efficiency.

[0033] Furthermore, in one embodiment, before step S10, the method further includes: When a trusted application running in the trusted execution environment TEE receives signature data sent by the algorithm authorization server, it verifies the legitimacy of the signature data; If the legitimacy verification passes, an authorization notification is sent to the main core.

[0034] Furthermore, in one embodiment, when the trusted application running in the trusted execution environment TEE receives the signature data issued by the algorithm authorization server, before the legitimacy verification of the signature data is performed, it also includes: The client application running in the rich execution environment REE sends a terminal model call request to the trusted application; When the trusted application receives the end-side model call request sent by the client application, it sends a signed random number to the client application; The client application sends the received signature random number to the cloud server; The cloud server checks whether the received signature random number is legal. If it is legal, it sends an authorization request to the algorithm authorization server; When the algorithm authorization server receives the authorization request, it signs the authorization field to obtain the signature data and sends the signature data to the cloud server; The cloud server sends the received signature data to the client application; The client application sends the received signed data to the trusted application.

[0035] Furthermore, in one embodiment, the terminal-side model calling method further includes: The algorithm authorization server obtains the identification information of the client application, and generates and saves the authorization record based on the time when the signature data is sent and the identification information.

[0036] Among them, the specific implementation of the above-mentioned end-side model calling method is basically the same as the various implementations of the end-side model calling system, and will not be repeated here one by one.

[0037] In a third aspect, an embodiment of the present application also provides a computer-readable storage medium.

[0038] The computer-readable storage medium of the present application stores an end-side model calling program, wherein when the end-side model calling program is executed by a processor, the steps of the end-side model calling method as described above are implemented.

[0039] Among them, the method implemented when the end-side model calling program is executed can refer to the various embodiments of the end-side model calling method of the present application, and will not be repeated here.

[0040] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are only for description and do not represent the advantages or disadvantages of the embodiments.

[0041] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit "first", "second" and "third" to different types.

[0042] In the description of the embodiments of the present application, "exemplary", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary", "for example" or "for example" is intended to present related concepts in a specific way.

[0043] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; the “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0044] In some processes described in the embodiments of the present application, multiple operations or steps that appear in a specific order are included, but it should be understood that these operations or steps may not be executed in the order in which they appear in the embodiments of the present application or in parallel, and the sequence number of the operation is only used to distinguish the different operations, and the sequence number itself does not represent any execution order. In addition, these processes may include more or fewer operations, and these operations or steps may be executed in sequence or in parallel, and these operations or steps may be combined.

[0045] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, disk, CD) as described above, and includes a number of instructions for a terminal device to execute the methods described in each embodiment of the present application.

[0046] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A terminal-side model calling system, characterized in that: The end-side model calling system includes a firewall and a neural network processor NPU, wherein the NPU internally integrates a master core, a slave core, an encryption engine, and a secure memory access controller SMAC, wherein: The main core is used to obtain the encrypted terminal model parameters when receiving the authorization notification; An encryption engine, used to decrypt the encrypted end-side model parameters to obtain the end-side model parameters, where the end-side model parameters are used to configure a neural network model running in the slave core; The firewall and SMAC are used to allocate secure memory for the neural network model operation so that the operation efficiency of the neural network model is greater than the preset efficiency.

2. The terminal-side model calling system according to claim 1, characterized in that: The end-side model calling system also includes a trusted application running in a trusted execution environment TEE, which is used to verify the legitimacy of the signature data when it is received from the algorithm authorization server; if the legitimacy verification passes, an authorization notification is sent to the main core.

3. The terminal-side model calling system according to claim 2, characterized in that: The end-side model calling system also includes a cloud server, an algorithm authorization server, and a client application running in a rich execution environment (REE), where: The client application is used to send a terminal model call request to the trusted application; The trusted application is used to send a signed random number to the client application when receiving a terminal model call request sent by the client application; The client application is used to send the received signature random number to the cloud server; The cloud server is used to detect whether the received signature random number is legal. If it is legal, it sends an authorization request to the algorithm authorization server; The algorithm authorization server is used to sign the authorization field to obtain signature data when receiving the authorization request, and send the signature data to the cloud server; The cloud server is used to send the received signature data to the client application; The client application sends the received signed data to the trusted application.

4. The terminal-side model calling system according to claim 3, characterized in that: The algorithm authorization server is also used to obtain the identification information of the client application, and generate and save the authorization record based on the time when the signature data is sent and the identification information.

5. A method for calling a terminal model, characterized in that: The terminal side model calling method includes: When receiving the authorization notification, the main core obtains the encrypted terminal-side model parameters; The encryption engine decrypts the encrypted end-side model parameters to obtain the end-side model parameters, which are used to configure the neural network model running in the slave core. Secure memory is allocated to the neural network model operation through the firewall and SMAC so that the operation efficiency of the neural network model is greater than the preset efficiency.

6. The terminal-side model calling method according to claim 5, characterized in that: When the main core receives the authorization notification, before obtaining the encrypted terminal-side model parameters, the method further includes: When a trusted application running in the trusted execution environment TEE receives signature data sent by the algorithm authorization server, it verifies the legitimacy of the signature data; If the legitimacy verification passes, an authorization notification is sent to the main core.

7. The terminal-side model calling method according to claim 6, characterized in that: When the trusted application running in the trusted execution environment TEE receives the signature data sent by the algorithm authorization server, before the legitimacy of the signature data is verified, it also includes: The client application running in the rich execution environment REE sends a terminal model call request to the trusted application; When the trusted application receives the end-side model call request sent by the client application, it sends a signed random number to the client application; The client application sends the received signature random number to the cloud server; The cloud server checks whether the received signature random number is legal. If it is legal, it sends an authorization request to the algorithm authorization server; When the algorithm authorization server receives the authorization request, it signs the authorization field to obtain the signature data and sends the signature data to the cloud server; The cloud server sends the received signature data to the client application; The client application sends the received signed data to the trusted application.

8. The terminal-side model calling method according to claim 7, characterized in that: The terminal-side model calling method further includes: The algorithm authorization server obtains the identification information of the client application, and generates and saves the authorization record based on the time when the signature data is sent and the identification information.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores an end-side model calling program, wherein when the end-side model calling program is executed by the processor, the steps of the end-side model calling method according to any one of claims 5 to 8 are implemented.

Images