Data processing method, electronic device, program product and storage medium
By partitioning memory space and managing access permissions in electronic devices, the problem of data tampering caused by CPU and NPU memory access permissions is solved, ensuring data integrity and confidentiality during NPU data processing and improving system security and stability.
Patent Information
- Application Number
- CN202411142167.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-19
- Publication Date
- 2026-03-03
AI Technical Summary
In electronic devices, both the CPU and NPU have access to memory. This can lead to malicious or legitimate programs malfunctioning and potentially tampering with data during the NPU's data processing phase, affecting the normal operation and security of the device.
The memory space is divided into three zones: a first data zone that can be read and written by TEE and REE, a second data zone that can be read and written by TEE and NPU, a third data zone that can be read and written by TEE and NPU, and a third data zone that is read-only by REE. Access permissions are managed through an access control module to ensure data isolation and security at different stages.
It effectively ensures the integrity and confidentiality of data during NPU data processing, improves the security and stability of the system, and prevents malicious or abnormal programs from tampering with data.
Smart Images
Figure CN121598429A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of security technology, and in particular to a data processing method, electronic device, program product, and storage medium. Background Technology
[0002] In the current hardware architecture of electronic devices, both the Central Processing Unit (CPU) and the Neural-network Processing Unit (NPU) have access to memory. Malicious programs on the CPU side accessing memory and tampering with data during the NPU data processing stage, or legitimate programs on the CPU side malfunctioning and incorrectly modifying data during the NPU data processing stage, can all affect the normal operation of electronic devices. Summary of the Invention
[0003] In view of the above problems, this application provides a data processing method, electronic device, program product, and storage medium to effectively ensure data integrity and confidentiality during NPU data processing, thereby improving system security and stability. The specific solution is as follows:
[0004] The first aspect of this application provides a data processing method using an electronic device. The electronic device is equipped with a Trusted Execution Environment (TEE) and a General Purpose Execution Environment (REE). The memory space of the electronic device is divided into at least a first data area, a second data area, and a third data area. Data stored in the first data area is accessed by modules of the TEE and REE for reading and writing. Data stored in the second data area is accessed by modules of the TEE and a Neural Processing Unit (NPU) for reading and writing. Data stored in the third data area is accessed by modules of the TEE and the NPU for reading and writing, and by modules of the REE for reading. The method includes:
[0005] In the TEE, the source data of the first application stored in the first data area is obtained and stored in the second data area; the source data of the first application is the source data of the data processing flow executed by the NPU; the first application is the application on the REE side.
[0006] The NPU obtains the source data of the first application stored in the second data area and performs a data processing flow on it;
[0007] The intermediate data of the source data of the first application is processed and stored in the second data area, and the data processing result of the source data of the first application is stored in the third data area.
[0008] In one possible implementation, after storing the source data of the first application in the second data area, the method further includes: triggering the NPU in the TEE to execute a data processing flow on the source data of the first application.
[0009] In one possible implementation, it also includes: in the REE, determining that the first application obtains source data, and storing the source data of the first application in the first data area.
[0010] In one possible implementation, before storing the source data of the first application in the first data area, the method further includes: signing the source data of the first application using an encryption key.
[0011] In one possible implementation, before storing the source data of the first application in the second data area, the following steps are also included:
[0012] Verify the signature of the source data for the first application;
[0013] The process of storing the source data of the first application in the second data area includes: storing the source data of the first application in the second data area when the source data of the first application is verified.
[0014] In one possible implementation, after obtaining the source data of the first application stored in the first data area within the TEE, it also includes:
[0015] The first application is validated for legitimacy. The validation includes: validation of the legitimacy of the APP identifier of the first application, validation of the replay attack of the processing request triggered by the first application to the NPU, and validation of the permission of the first application to trigger the NPU function.
[0016] When the first application passes the legitimacy verification, the source data of the first application is stored in the second data area.
[0017] One possible implementation also includes:
[0018] In the TEE, the data processing results of the source data of the first application stored in the third data area are signed based on the encryption key.
[0019] One possible implementation also includes:
[0020] In the REE, the data processing results of the source data of the first application stored in the third data area are obtained;
[0021] Verify the signature of the data processing results of the source data of the first application;
[0022] When the data processing result of the source data of the first application is verified and approved, the first application executes business based on the data processing result of the source data of the first application.
[0023] A second aspect of this application provides an electronic device, including at least one processor and a memory connected to the processor, wherein: the processor includes: a central processing unit (CPU) and a neural network processor (NPU), the CPU of the electronic device is deployed with a Trusted Execution Environment (TEE) and a General Purpose Execution Environment (REE), and the storage space of the memory is divided into at least a first data area, a second data area and a third data area, the data stored in the first data area is used for read and write access by modules of the TEE and the REE, the data stored in the second data area is used for read and write access by modules of the TEE and the neural network processor (NPU), and the data stored in the third data area is used for read and write access by modules of the TEE and the neural network processor (NPU), as well as read access by modules of the REE;
[0024] Memory is also used to store computer programs;
[0025] The processor is used to execute computer programs to enable electronic devices to implement the data processing method described in the first aspect or any implementation thereof.
[0026] A third aspect of this application provides a computer program product including computer-readable instructions that, when executed on an electronic device, cause the electronic device to implement the data processing method described in the first aspect or any implementation thereof.
[0027] The fourth aspect of this application provides a computer storage medium carrying one or more computer programs, which, when executed by an electronic device, enable the electronic device to perform the data processing method described in the first aspect or any implementation thereof.
[0028] Using the above technical solution, in the data processing method provided by this application, the storage space of the electronic device's memory is divided into at least a first data area, a second data area, and a third data area. The first data area can be read and written by TEE and REE, the second data area can be read and written by TEE and NPU, and the third data area can be read and written by TEE and NPU, as well as read by REE. The source data of the first application is stored in the second data area within the TEE. The intermediate data of the data processing flow executed by the NPU on the source data of the first application is also stored in the second data area. Since the second data area can be read and written by both the TEE and the NPU, it can prevent the source data of the first application and the intermediate data of the NPU from being tampered with by malicious programs on the REE side. Of course, the first application cannot modify the intermediate data of the NPU on the REE side either. It can also prevent the first application from making incorrect modifications to the intermediate data of the NPU due to an anomaly. Furthermore, the data processing result of the source data of the first application is stored in the third data area. Since the third data area can be read-only by the REE, it can prevent the data processing result of the NPU from being tampered with by malicious programs on the REE side, and it can also prevent the first application from making incorrect modifications to the data processing result of the NPU. In this way, the integrity and confidentiality of the data during the data processing of the NPU are effectively guaranteed, thereby improving the security and stability of the system. Attached Figure Description
[0029] The features, advantages, and aspects of the embodiments of this application will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.
[0030] Figure 1 An architectural diagram of an electronic device provided in an embodiment of this application;
[0031] Figure 2 This application provides a spatial partitioning diagram of a memory in an electronic device.
[0032] Figure 3 An illustration of an APP access whitelist stored on the TEE side and stored on the REE side in an electronic device, provided as an embodiment of this application;
[0033] Figure 4 A flowchart illustrating the data processing method provided in the embodiments of this application;
[0034] Figure 5 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation
[0035] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.
[0036] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.
[0037] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.
[0038] In current electronic device hardware architectures, the Central Processing Unit (CPU) can directly send instructions to the Neural-network Processing Unit (NPU), and both the CPU and NPU typically have access to memory. While this design simplifies hardware architecture and data transmission to some extent, it also introduces certain data security risks and program malfunction risks.
[0039] The data security risks are as follows:
[0040] Both the CPU and NPU have access to memory. This means that any program running on the CPU, whether legitimate or malicious, could potentially access and modify data processed by the NPU, which could lead to the following risks:
[0041] (1) Risk of source data tampering
[0042] Malicious programs can access memory through the CPU and tamper with the raw data (i.e., source data) processed by the NPU. This can cause the NPU to produce incorrect results during processing, thereby affecting the normal operation of electronic devices.
[0043] (2) Risk of intermediate result tampering
[0044] During NPU data processing, intermediate results are often stored in memory. Malicious programs can access and tamper with these intermediate results, causing the NPU to generate incorrect processing results. These tampering actions can compromise data integrity, lead to functional failures, and even trigger more serious security issues.
[0045] (3) Operational hijacking risk
[0046] Malicious programs can trigger NPU operations through the CPU. For example, a malicious program can send incorrect commands or parameters to the NPU, causing it to perform unsafe operations or even triggering system crashes. This type of operation hijacking seriously threatens the stability and security of electronic devices.
[0047] The following are the program's abnormal risks:
[0048] Program malfunctions can lead to security risks and data corruption. For example, in some cases, a legitimate program may accidentally access or modify data and commands processed by the NPU due to programming errors or system failures. Such program malfunctions can not only affect the normal operation of electronic devices but also potentially lead to data leaks and security vulnerabilities.
[0049] Based on this, the present application provides a data processing method that effectively ensures the integrity and confidentiality of data during the NPU data processing process, improves the security and stability of the system, and avoids data security risks and program abnormality risks in electronic devices.
[0050] In the embodiments of this application, such as Figure 1 As shown, the electronic device 100 includes a CPU 101 and an NPU 102. The CPU 101 includes a Rich Execution Environment (REE) and a Trusted Execution Environment (TEE). REE and TEE are two parallel system operating environments. This application embodiment does not limit the number of these two types of execution environments. Furthermore, TEE and REE are merely names to distinguish the two types of execution environments; with technological advancements, TEE and REE can be replaced with other names that have the same function.
[0051] REE aims to provide an open system operating environment. The security performance of REE is lower than that of TEE. It can usually run a large number of applications (APP), and the computing and storage resources configured in REE are more than those configured in TEE.
[0052] REE includes a rich execution environment operating system (REE OS) and client applications (CA). The rich execution environment operating system (REE OS) can also be referred to as a Rich OS.
[0053] TEE aims to ensure the security, confidentiality, and integrity of code and data loaded into this environment. TEE provides an isolated execution environment with security features including: isolated execution, integrity of trusted applications, confidentiality of trusted data, and secure storage.
[0054] TEE includes a Trusted Execution Environment Operating System (TEE OS), which provides trusted security services to CAs (such as fingerprint verification, password verification, and facial recognition). These security services can run on the TEE OS as trusted applications (TAs). The TEE OS can also be called a Trusted OS.
[0055] The NPU102 is a neural network (NN) processor that, by drawing inspiration from the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, rapidly processes input information and can continuously learn on its own. NPUs can be used to enable applications such as intelligent cognition in electronic devices.
[0056] For example, in one application scenario, the electronic device is a terminal device in self-driving (e.g., an autonomous vehicle), a terminal device in intelligent driving, etc. The NPU102 can recognize road image information in intelligent driving functions. Through the recognition results of the NPU102, the electronic device can complete the planning of BEV (Bird's-eye-view).
[0057] During the system startup phase of an electronic device, it is necessary to configure the permissions of the NPU and the access permissions of the memory. In some embodiments, the memory of the electronic device may refer to physical memory.
[0058] like Figure 1As shown, during the system startup phase of the electronic device, the access permissions of the NPU are configured in the Trusted OS to ensure that the CPU can only access the NPU from the TEE side and cannot access the NPU from the REE side, thus avoiding the risk of directly accessing the NPU from the REE side.
[0059] In some embodiments, the electronic device includes an access control module, such as the TrustZone Address SpaceController 400 (TZC400), which configures permissions for the various hardware components it manages. Typically, the access control module can be accessed within a trusted execution environment. Therefore, in a Trusted OS, the access control module is configured to configure permissions for the NPU.
[0060] The access control module configures the NPU's access permissions as follows: on the CPU side, access to the NPU can only be performed from the TEE side, and access control to the NPU cannot be performed from the REE side. Subsequently, the electronic device can listen for NPU access commands. If the access command originates from the TEE side, it can be executed; if it originates from the REE side, it can be discarded.
[0061] like Figure 2 As shown, during the initialization and startup phase, TEEOS configures and partitions the memory 103 used for interaction between the CPU and NPU, dividing the memory 103 into at least three types:
[0062] In the first data area, both TEE and REE can be read and write access controlled, and the REE-side APP inputs source data for NPU processing.
[0063] The second data area is for TEE and NPU read / write access control. This data area is used for TEE to input data to NPU and for NPU processing (i.e., to store intermediate data during NPU data processing). The REE side does not have read / write control permissions for this data area.
[0064] The third data area contains read / write access control for TEE and NPU, and read-only access for REE. This data area is used by NPU to return neural network data processing results, TEE data is encrypted, and REE obtains complete result data.
[0065] In some embodiments, an access control module, such as a TrustZone Address Space Controller 400 (TZC400), can also configure access permissions to the memory 103. The access control module can divide the access address space of the memory 103 into at least three segments: one corresponding to the first data area, another to the second data area, and another to the third data area. Of course, the memory 103 can also have other types of data areas; this embodiment does not impose specific limitations on this.
[0066] It should be noted that dividing the memory 103 into multiple regions with different read and write permissions achieves multi-level memory isolation, enabling strict isolation of data at different data processing stages of the NPU, and ensuring the security and integrity of data during processing.
[0067] After the access control module configures the access permissions for the memory, the electronic device can also listen for read and write commands from the memory. If the command originates from the TEE or REE, read and write commands for the first data area are allowed to be executed. If the read and write command originates from the TEE or NPU, the command is allowed to be executed. If the command originates from the REE, the command is not allowed to be executed and can be discarded. If the read and write command originates from the TEE or NPU, the command is allowed to be executed. If the command originates from the REE, read commands are allowed to be executed, but write commands are not allowed to be executed.
[0068] The TEE side of the electronic device also stores a whitelist of APP access on the REE side. APPs on the REE side in the whitelist are legitimate APPs and can access the NPU. That is, the NPU can process the source data of APPs on the REE side in the whitelist.
[0069] In some embodiments, such as Figure 3 As shown, the APP access whitelist on the REE side includes: the legitimate APP_id, cmd_counter, encryption key, and other legitimacy verification information such as function permissions; among which:
[0070] APP_id, also known as APP identifier, is used to verify the legitimacy of an APP.
[0071] `cmd_counter` is used to prevent replay attacks. Electronic devices can use replay attack mechanisms to listen for TEE's access commands to the NPU and not respond to repeated NPU access commands, thus preventing problems during the operation of the electronic device.
[0072] Encryption keys, such as asymmetric encryption keys, are used to verify and check the source data of the APP and the result data processed by the NPU.
[0073] Function permissions are used to restrict the NPU access functions of different apps. An NPU can have multiple functions. For an app, function permissions control one or more functions of the NPU that the app can access. The app cannot use the functions of the NPU that are not within the scope of the function permissions. In other words, the electronic device does not provide the app with NPU functions that are not within its function permissions.
[0074] For example, such as Figure 3 As shown, the REE side includes n apps, where n is greater than or equal to 1. The TEE side stores the REE side's app access whitelist, which includes the legitimacy verification information for x apps, where x is a positive integer less than or equal to n. Taking APP1 and APP3 on the REE side as examples, the legitimacy verification information for APP1 in the REE side's app access whitelist stored on the TEE side indicates that APP1 is legitimate and can use some or all of the NPU's functions, while APP3 is illegitimate and cannot use the NPU's functions.
[0075] Based on the configuration of NPU permissions and memory access permissions, as well as the APP access whitelist on the REE side, the electronic device can execute the data processing method provided in the embodiments of this application.
[0076] The following text combines Figure 4 The data processing methods applicable to the embodiments of this application are described below.
[0077] like Figure 4 As shown, the data processing method includes the following steps:
[0078] The S100 and TEE sides complete the NPU permission configuration and memory space allocation.
[0079] Before an electronic device executes the data processing method provided in the embodiments of this application, it needs to complete the NPU permission configuration and memory space allocation configuration on the TEE side. The configuration process can be found in the foregoing content and will not be repeated here.
[0080] S101, the REE side stores the source data of the first application in the first data area and signs it.
[0081] The first application can be understood as an application on the REE side, such as an intelligent driving application. After the first application is started and running, in some application scenarios, the first application can generate data that needs to be processed by the NPU, which is referred to as the source data of the first application in this embodiment.
[0082] The REE side can store the source data of the first application into the first data area of the memory. The REE side can be understood as the Rich OS on the REE side of the electronic device. The Rich OS can obtain the source data of the first application and write it into the first data area of the memory.
[0083] In some embodiments, in order to ensure the security of the source data of the first application, the first application can sign the source data of the first application using an encryption key, and then the REE side stores the signed source data of the first application in the first data area.
[0084] In other embodiments, after Rich OS obtains the source data of the first application, it signs the source data of the first application using an encryption key, and then Rich OS stores the signed source data of the first application in the first data area.
[0085] It should be noted that the encryption key used to encrypt the source data of the first application can be the same as the encryption key of the first application in the APP access whitelist stored on the TEE side, or it can be combined with the encryption key of the first application in the APP access whitelist stored on the TEE side to form an asymmetric encryption key.
[0086] The source data of the first application stored in the first data area is encrypted, which can ensure the security of the source data of the first application and prevent the source data of the first application from being tampered with by malicious programs.
[0087] It should also be noted that malicious programs typically reside on the REE side, while applications on the TEE side are usually secure. Furthermore, REE-side applications have a higher probability of erroneously modifying data and commands processed by the NPU. Currently, TEE-side applications generally do not use the NPU to process their source data. Even if a TEE-side application entrusts its source data to the NPU, it is relatively unlikely that the TEE-side application will erroneously modify the data and commands processed by the NPU.
[0088] Based on this, in some application scenarios, the data processing method provided in this application embodiment can be understood as being applied to the REE-side application, where the NPU processes its source data. Of course, when the TEE-side application utilizes the NPU processor's source data, on the TEE side, the application's source data can be directly stored in the second data area, and then the following steps S106 to S109 are executed. Afterward, the TEE-side application obtains the data processing results from the third data area and uses them according to business needs.
[0089] S102, The REE side notifies the TEE of the service processing NPU processing request.
[0090] After the source data of the first application is stored in the first data area, the REE side can send a notification message to the TEE side. This notification message is used to notify the service in the TEE to handle the NPU processing request. The service in the TEE can be understood as the service that interfaces with the NPU on the TEE side, which is a functional module, and this application does not limit it. In some embodiments, the service in the TEE can refer to Trusted OS, and of course, it can refer to the NPU driver service in Trusted OS, which can be used to drive the NPU to execute the data processing flow.
[0091] Services in the TEE can perform the following steps S103 to S105 based on NPU processing requests.
[0092] S103, the TEE side performs a legality verification on the first application.
[0093] As mentioned above, the TEE side stores the APP access whitelist of the REE side, and the TEE side performs legality verification on the first application based on the APP access whitelist of the REE side.
[0094] In some embodiments, the APP access whitelist on the REE side includes: the APP_id, cmd_counter, encryption key, and function permissions of the legitimate APP. Based on this, the TEE side performs legitimacy verification on the first application by: performing whitelist verification on the APP_id of the first application, performing cmd_counter verification on the first application, and performing whitelist verification on the function permissions of the first application.
[0095] The TEE side searches for the APP_id of the first application in the APP access whitelist on the REE side. If the APP_id of the first application belongs to the APP access whitelist on the REE side, then the APP_id of the first application passes the whitelist verification; otherwise, the APP_id of the first application fails the whitelist verification.
[0096] In this step, accessing the whitelist through the REE-side APP can prevent malicious programs from triggering NPU operations, causing the NPU to perform unsafe operations, or even causing system crashes.
[0097] The TEE side uses a replay attack mechanism to verify whether the NPU processing request corresponding to the first application is a duplicate NPU processing request. If the NPU processing request corresponding to the first application is not a duplicate NPU processing request, the first application passes the cmd_counter verification; otherwise, it fails the cmd_counter verification. Here, the NPU processing request corresponding to the first application can be understood as the NPU processing request mentioned in step S102, which can be understood as the NPU processing request corresponding to the source data of the first application processed by the NPU.
[0098] In this step, the cmd_counter mechanism is used to prevent replay attacks and ensure the uniqueness and legitimacy of each data processing request from the NPU.
[0099] The TEE side verifies the NPU processing request corresponding to the first application in the whitelist based on the APP access on the REE side. That is, it verifies whether the NPU processing request corresponding to the first application is allowed by the function permission of the first application. If it is allowed, the function permission of the first application passes the whitelist verification; otherwise, it fails.
[0100] In this step, by setting the NPU function permissions of the REE-side application through a whitelist, the operation of the NPU can be effectively protected and blocked.
[0101] It should be noted that if the TEE side determines that the first application's APP_id passes the whitelist verification, the first application passes the cmd_counter verification, and the first application's function permissions pass the whitelist verification, then the first application passes the legality verification, meaning the first application is legal; otherwise, it fails the legality verification, meaning the first application is illegitimate.
[0102] In some embodiments, the TEE side performs legality verification on the first application, which can be understood as the service in the TEE performing legality verification on the first application.
[0103] S104. When the first application is valid, the TEE side verifies the signature of the source data of the first application in the first data area.
[0104] As can be seen from step S101: the source data of the first application stored in the first data area is encrypted. The TEE can decrypt the encrypted source data of the first application based on the key of the first application, that is, verify the signature.
[0105] In some embodiments, the source data of the first application is encrypted using the public key of an asymmetric encryption key, and the TEE side decrypts the encrypted source data of the first application using the private key of the asymmetric encryption key. In this case, the TEE side decrypts the encrypted source data of the first application using the encryption key of the first application in the APP access whitelist on the REE side, which can be understood as the private key of the asymmetric encryption key.
[0106] In other embodiments, the encryption key for the source data of the first application and the decryption key on the TEE side are the same key.
[0107] In some embodiments, the TEE side verifies the signature of the source data of the first application in the first data area. This can be understood as the service in the TEE verifying the signature of the source data of the first application in the first data area.
[0108] S105. When the verification is successful, the TEE side stores the source data of the first application into the second data area of the memory.
[0109] If the TEE side can successfully decrypt the encrypted source data of the first application, the verification is successful. The TEE side then stores the decrypted source data of the first application in the second data area of the memory. The REE side does not have read / write control permissions for the second data area; therefore, the decrypted source data of the first application is relatively secure.
[0110] In some embodiments, the TEE side stores the source data of the first application in the second data area. This can be understood as the service in the TEE storing the source data of the first application in the second data area.
[0111] In some embodiments, steps S103 and S104 may be omitted, and on the TEE side, the source data of the first application stored in the first data area may be directly stored in the second data area.
[0112] S106, TEE side triggers NPU to perform data processing.
[0113] Based on steps S103 and S104, the TEE side determines that the first application has the right to use the NPU function and that the source data of the first application has been legally decrypted. The TEE side can then trigger the NPU to perform data processing on the source data of the first application. In this way, the NPU can be controlled and run by a legitimate program, preventing malicious programs from triggering NPU operations and causing the NPU to perform unsafe operations that could lead to system crashes and threaten the stability and security of the electronic device system.
[0114] In some embodiments, the TEE side can exit the process if it verifies that the first application is illegitimate based on step S103, or if it cannot legally decrypt the source data of the first application through step S104.
[0115] In some embodiments, the TEE side can send control commands to the NPU102, which are used to trigger the NPU to perform data processing. In some embodiments, services in the TEE side send control commands to the NPU102.
[0116] S107 and NPU102 obtain the source data of the first application from the second data area, process the data, and store the intermediate results of the data processing in the second data area.
[0117] The NPU102 has read / write control permissions for the second data area. Therefore, in response to a trigger from the TEE side, the NPU102 retrieves the source data of the first application from the second data area. After retrieving the source data, the NPU102 processes it. During the data processing, the NPU102 generates intermediate results, which can be stored in the second data area to ensure the security of the intermediate results and prevent them from being tampered with.
[0118] Of course, since the trigger received by NPU102 originates from the TEE side, the trigger is allowed to be responded to by the NPU. In some cases, if NPU102 receives a trigger command from the REE side, the NPU will not respond to the command. In some cases, even if the REE side receives a trigger command from NPU102, the trigger command cannot be directly sent to NPU102.
[0119] Since the NPU102 only responds to trigger commands from the TEE side, it can avoid being triggered by malicious programs on the REE side to perform unsafe operations and cause system crashes.
[0120] The data processing of the source data of the first application by NPU102 can be understood as conventional technology, and this application does not limit it.
[0121] S108 and NPU102 store the data processing results in the third data area of the memory.
[0122] After NPU102 completes data processing on the source data of the first application, it obtains the data processing result. NPU102 stores the data processing result in the third data area of the memory. Since only TEE and NPU have read and write access control to the third data area, and REE only has read-only permissions, the data processing result stored in the third data area also has high security and can prevent the data processing result from being tampered with.
[0123] S109 and NPU102 notify the TEE side that data processing is complete.
[0124] After NPU102 completes data processing on the source data of the first application, it can notify the TEE side that data processing is complete. In some embodiments, NPU can send a notification message to the TEE side to notify it that data processing is complete. In some embodiments, NPU can send a notification message to the services within the TEE.
[0125] After receiving the notification message from NPU102, the TEE side can execute the following steps S110 and S111, and the execution order of steps S110 and S111 is not restricted. Figure 4As shown, the TEE side can also execute step S111 first, then step S110, or execute steps S110 and S111 in parallel.
[0126] S110 and TEE sides add signatures to the data processing results in the third data area.
[0127] In some embodiments, after receiving a notification from the NPU102, the TEE side learns that the source data of the first application has been processed and can obtain the data processing result from the third data area. After obtaining the data processing result of the source data of the first application, the TEE side encrypts the data processing result, i.e., signs or affixes a signature. Then, the TEE side stores the encrypted data processing result in the third data area.
[0128] In other embodiments, the TEE side can directly encrypt the data processing results in the third data area.
[0129] It should be noted that the key used by the TEE side to sign the data processing results in the third data area can be the encryption key corresponding to the first application in the whitelist accessed by the APP on the REE side.
[0130] Of course, the TTE side may also choose not to sign the data processing results stored in the third data area. In this case, after the first application obtains the data processing results through the following step S113, there is no need for signature verification.
[0131] S111, the TEE side notifies the REE side that data processing is complete.
[0132] After receiving the notification from NPU102, the TEE can notify the REE that data processing is complete. In some embodiments, the TEE can send a notification message to the REE to inform it that data processing is complete. In some embodiments, services in the TEE can send notification messages to the Rich OS of the REE.
[0133] S112, the REE side obtains the data processing results from the third data area.
[0134] The REE side grants read-only access to the data in the third data area. Therefore, the REE side can obtain the data processing results of the source data of the first application from the third data area. Of course, the data processing results are encrypted.
[0135] In some embodiments, the Rich OS of the REE obtains the data processing results from the third data area and then sends the data processing results to the first application.
[0136] S113. The REE side verifies and signs the data processing results according to business needs before using them.
[0137] The first application on the REE side decrypts the data processing results, that is, after verification, it uses the data processing results based on business needs. For example, the first application is an intelligent driving application, which can complete the planning of BEV based on the data processing results.
[0138] In some embodiments, the first application decrypts the data processing result based on the decryption key of the encryption key corresponding to the first application in the APP access whitelist on the REE side.
[0139] In the embodiments of this application, the above steps S102, S109 and S111 are all used to implement interactive notification between REE, TEE and NPU. In some embodiments, steps S102, S109 and S111 may not be executed, or only partially executed.
[0140] The above describes a data processing method provided by an embodiment of this application. The following will describe the apparatus for performing the above data processing method. In some embodiments, Figure 1 The electronic device 100 shown can be used as a device to perform the above-described data processing method.
[0141] Figure 5 A schematic diagram of the structure of an electronic device 100 is provided.
[0142] The electronic device can be a mobile phone, tablet, computer, virtual reality (VR) terminal device, augmented reality (AR) terminal device, terminal device in industrial control, terminal device in self-driving (e.g., driverless car), terminal device in intelligent driving, terminal device in smart grid (e.g., set-top box, smart TV), terminal device in smart city, terminal device in smart home, etc.
[0143] like Figure 5 As shown, the electronic device may include a bus 501, a processor 502, a communication interface 503, and a memory 504. The processor 502, the memory 504, and the communication interface 503 communicate with each other via the bus 501.
[0144] Bus 501 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be divided into address buses, data buses, control buses, etc.
[0145] Processor 502 includes at least a central processing unit (CPU) and a neural-network processing unit (NPU), and may also include any one or more processors such as a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP). As mentioned above, the CPU includes a rich execution environment (REE) and a trusted execution environment (TEE).
[0146] Memory 504 may include volatile memory, such as random access memory (RAM). Memory 704 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD). As described above, the storage space of memory 504 is divided into at least a first data area, a second data area, and a third data area. Data stored in the first data area is used for read and write access by TEE and REE modules; data stored in the second data area is used for read and write access by TEE modules and NPU; and data stored in the third data area is used for read and write access by TEE modules and NPU, as well as read access by REE modules.
[0147] The memory 504 can be used to store software code related to the data processing method, and the processor 502 can execute the steps of the data processing method and schedule other units to achieve the corresponding functions.
[0148] Figure 5The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.
[0149] This application also provides a computer program product including computer-readable instructions, which, when executed on an electronic device, cause the electronic device to implement any of the data processing methods provided in this application.
[0150] The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, training equipment, or data center to another website, computer, training equipment, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means.
[0151] This application also provides a computer-readable storage medium that carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement any of the data processing methods provided in this application.
[0152] Computer-readable storage media can be any available medium that a computer can store, or a data storage device such as a training device or data center that integrates one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).
Claims
1. A data processing method, characterized in that, An application electronic device is provided, the electronic device deploying a Trusted Execution Environment (TEE) and a General Purpose Execution Environment (REE). The memory space of the electronic device is divided into at least a first data area, a second data area, and a third data area. Data stored in the first data area is used for read and write access by modules of the TEE and the REE. Data stored in the second data area is used for read and write access by modules of the TEE and a Neural Processing Unit (NPU). Data stored in the third data area is used for read and write access by modules of the TEE and the NPU, as well as for read access by modules of the REE. The method includes: In the TEE, the source data of the first application stored in the first data area is obtained and stored in the second data area; the source data of the first application is the source data of the data processing flow executed by the NPU; the first application is the application on the REE side. The NPU obtains the source data of the first application stored in the second data area and performs a data processing flow on it; The intermediate data of the data processing flow of the source data of the first application is stored in the second data area, and the data processing result of the source data of the first application is stored in the third data area.
2. The data processing method according to claim 1, characterized in that, After storing the source data of the first application in the second data area, the method further includes: Within the TEE, the NPU is triggered to execute a data processing flow on the source data of the first application.
3. The data processing method according to claim 1, characterized in that, Before acquiring the source data of the first application stored in the first data area in the TEE, the following steps are also included: In the REE, it is determined that the first application obtains source data, and the source data of the first application is stored in the first data area.
4. The data processing method according to claim 3, characterized in that, Before storing the source data of the first application in the first data area, the method further includes: The source data of the first application is signed using the encryption key.
5. The data processing method according to claim 4, characterized in that, Before storing the source data of the first application in the second data area, the method further includes: Verify the signature of the source data of the first application stored in the first data area; The step of storing the source data of the first application in the second data area includes: storing the source data of the first application in the second data area when the source data of the first application passes the signature verification.
6. The data processing method according to any one of claims 1 to 5, characterized in that, After obtaining the source data of the first application stored in the first data area in the TEE, the process further includes: The first application is validated for legitimacy, which includes: validation of the legitimacy of the APP identifier of the first application, validation of the replay attack of the processing request of the first application to trigger the NPU, and validation of the permission of the first application to trigger the NPU function. When the first application passes the legality verification, the source data of the first application is stored in the second data area.
7. The data processing method according to any one of claims 1 to 5, characterized in that, Also includes: In the TEE, the data processing results of the source data of the first application stored in the third data area are signed based on the encryption key.
8. The data processing method according to claim 7, characterized in that, Also includes: In the REE, the data processing results of the source data of the first application stored in the third data area are obtained; Verify the signature of the data processing results of the source data of the first application; When the data processing result of the source data of the first application is verified and approved, the first application executes business based on the data processing result of the source data of the first application.
9. An electronic device, characterized in that, The device includes at least one processor and a memory connected to the processor, wherein: the processor includes a central processing unit (CPU) and a neural network processor (NPU); the CPU of the electronic device is deployed with a Trusted Execution Environment (TEE) and a General Purpose Execution Environment (REE); the storage space of the memory is divided into at least a first data area, a second data area, and a third data area; the data stored in the first data area is used for read and write access by modules of the TEE and the REE; the data stored in the second data area is used for read and write access by modules of the TEE and the NPU; and the data stored in the third data area is used for read and write access by modules of the TEE and the NPU, as well as for read access by modules of the REE. The memory is also used to store computer programs; The processor is used to execute the computer program to enable the electronic device to implement the data processing method as described in any one of claims 1 to 8.
10. A computer program product, characterized in that, It includes computer-readable instructions that, when executed on an electronic device, cause the electronic device to perform the data processing method as described in any one of claims 1 to 8.
11. A computer storage medium, characterized in that, The storage medium carries one or more computer programs that, when executed by an electronic device, enable the electronic device to implement the data processing method as described in any one of claims 1 to 8.