Display method of trusted user interface and electronic device

Abstract

The present application discloses a method for displaying a trusted user interface and an electronic device, which is applied to an electronic device with a folding screen, and the method includes: in response to a first operation of the user on the electronic device, changing the folding screen from a first display state to a second display state; displaying the user interface UI of the CA on the screen corresponding to the second display state; in response to a second operation of the user on the UI of the CA, triggering the display of the trusted user interface TUI corresponding to the CA on the screen corresponding to the second display state, and the TUI is adapted to the screen corresponding to the second display state. The present application can realize that the trusted user interface (TUI) of the TA corresponding to the CA is automatically adapted to the screen changes of the folding screen, thereby improving the user experience.

Description

Technical Field

[0001] The present invention relates to the field of electronic devices, and in particular to a display method of a trusted user interface (TUI) and an electronic device. Background Art

[0002] Nowadays, electronic devices such as mobile phones have been widely popularized and used. On the one hand, mobile payment or online banking or mobile banking has become an important payment method for people's daily consumption. Mobile payment refers to a service method in which users use mobile terminals (usually mobile phones) to pay for the goods or services they consume. While mobile payment brings convenience to users, its security has always been a concern. Mobile payment applications usually run in an open rich execution environment (REE). The applications in REE are also called client applications (CA). CA runs on terminal operating systems such as Android® and iOS® on general-purpose processors. This open environment provides a channel for information leakage and malware propagation. In order to improve the security requirements of mobile payment, the Open Mobile Terminal Platform (OMTP) proposed the concept of Trusted Execution Environment (TEE). TEE is an independent operating environment that runs outside of REE and is isolated from REE. Applications in TEE are specially customized trusted applications (TA), and TA can access the hardware and software resources of TEE through the internal interface of TEE. The CA in REE cannot directly access the hardware and software resources of TEE. Only after passing TEE identity authentication can the CA in REE call TEE resources or services, such as secure storage, secure display / input, etc., through the Application Programming Interface (API) provided by TEE. In this way, in the mobile payment scenario, if the input and display of sensitive information is involved, the CA on the REE side can call the secure display / input TA in TEE to display the Trusted User Interface (TUI) that complies with the GP specification, so that users can protect their sensitive information through TUI, such as the input of the Personal Identification Number (PIN) and the confirmation of transaction information. After the TUI is displayed, the entire screen display area is taken over by TEE, and REE is completely blocked from accessing the display area, thereby preventing the CA in REE from malicious programs eavesdropping and stealing user sensitive information.

[0003] On the other hand, with the development of screen technology, the display screens of electronic devices are becoming larger and larger to provide users with richer information and better user experience. However, if the display screen of an electronic device is too large, it will be inconvenient to carry. Therefore, electronic devices with foldable screens (such as foldable screen mobile phones) proposed in recent years are an important development direction of electronic devices.

[0004] In actual applications, when the screen state of the folding screen (such as the folding state) changes, the Android side will broadcast to notify the application CA in the REE, and the user interface (UI) of CA will achieve adaptive screen display adaptation through the processing of the graphics processing unit (GPU). However, since TEE and Android are isolated systems, and TEE usually does not have complex display systems such as GPU, TEE cannot perceive the screen state changes of the folding screen. This will cause the trusted user interface (TUI) of the TA corresponding to CA to be unable to automatically adapt to the screen changes of the folding screen in scenarios such as mobile payment, which brings inconvenience to users. Summary of the invention

[0005] The embodiments of the present invention provide a display method and electronic device for a trusted user interface, which can enable the trusted user interface (TUI) of the TA corresponding to the CA to automatically adapt to the screen changes of the folding screen, thereby improving the user experience.

[0006] In a first aspect, an embodiment of the present invention provides a method for displaying a trusted user interface, the method being applied to an electronic device having a folding screen, the folding screen comprising at least two screens, a client application CA running on the electronic device, the method comprising: in response to a first operation of the user on the electronic device, changing the folding screen from a first display state to a second display state; wherein the first display state indicates that at least one of the at least two screens is in a display state, the second display state indicates that at least one of the at least two screens is in a display state, and the screen corresponding to the first display state is different from the screen corresponding to the second display state; displaying the user interface UI of the CA on the screen corresponding to the second display state; in response to a second operation of the user on the UI of the CA, triggering display of the trusted user interface TUI corresponding to the CA on the screen corresponding to the second display state, and the TUI is adapted to the screen corresponding to the second display state.

[0007] It can be seen that in the embodiment of the present invention, for an electronic device with a foldable screen, when the display state of the foldable screen where the application service is located changes, the embodiment of the present invention can realize both the adaptive display of CA's UI on the foldable screen and the adaptive display of TA's TUI on the foldable screen, which can solve the pain points of foldable screen users and greatly improve the user experience.

[0008] Based on the first aspect, in a possible embodiment, the first operation is an operation of unfolding the folding screen into a full-screen display; the first display state indicates that one of the at least two screens is in a display state, and the second display state indicates that the at least two screens are in a display state.

[0009] Based on the first aspect, in a possible embodiment, the first operation is an operation of folding the folding screen into a single-screen display; the first display state indicates that the at least two screens are in a display state, and the second display state indicates that one of the at least two screens is in a display state.

[0010] Based on the first aspect, in a possible embodiment, the first operation is an operation of flipping the electronic device to switch the folding screen from a first single-screen display to a second single-screen display.

[0011] The first display state indicates that a first screen among the at least two screens is in a display state, and the second display state indicates that a second screen among the at least two screens is in a display state.

[0012] Based on the first aspect, in a possible embodiment, after triggering display of the TUI of the CA on the screen corresponding to the second display state, the method further includes:

[0013] In response to a third operation of the user on the electronic device, determining that the folding screen is changed from the second display state to a third display state, wherein the third display state indicates that at least one of the at least two screens is in a display state, and a screen corresponding to the third display state is different from a screen corresponding to the second display state;

[0014] exiting the TUI in the screen corresponding to the third display state, and displaying the UI of the CA;

[0015] In response to a fourth operation of the user on the UI of the CA, the TUI of the CA is triggered to be displayed on the screen corresponding to the third display state, and the TUI is adapted to the screen corresponding to the third display state.

[0016] The third operation is, for example, an operation of folding the foldable screen (folding from the full screen to a single screen), or an operation of unfolding the foldable screen (expanding from a single screen to a full screen), or an operation of flipping the electronic device (flipping from one single-screen display to another single-screen display).

[0017] Based on the first aspect, in a possible embodiment, the fourth operation and the second operation are the same operation.

[0018] Based on the first aspect, in a possible embodiment, the screen corresponding to the third display state is the same as the screen corresponding to the first display state;

[0019] Alternatively, the screen corresponding to the third display state is different from the screen corresponding to the first display state.

[0020] Based on the first aspect, in a possible embodiment, the adaptation of the TUI to the screen corresponding to the second display state means that: the TUI occupies the entire screen on the screen corresponding to the second display state.

[0021] Based on the first aspect, in a possible embodiment, the adaptation of the TUI to the screen corresponding to the second display state means that: the TUI occupies a part of the screen on the screen corresponding to the second display state.

[0022] Based on the first aspect, in a possible embodiment, the electronic device is deployed with a rich execution environment REE module and a trusted execution environment TEE module, the CA runs in the REE module; the trusted application TA corresponding to the CA runs in the TEE module;

[0023] The step of displaying the UI of the CA on the screen corresponding to the second display state includes:

[0024] The UI of the CA is displayed on the screen corresponding to the second display state through the REE module.

[0025] The triggering, in response to the second operation of the user on the UI of the CA, to display the TUI corresponding to the CA on the screen corresponding to the second display state comprises:

[0026] Through the TEE module, in response to the second operation of the user on the UI of the CA, a trigger is provided to display the TUI of the TA in the TEE module on the screen corresponding to the second display state.

[0027] Based on the first aspect, in a possible embodiment, in response to a first operation of the user on the electronic device, determining that the folding screen changes from the first display state to the second display state includes:

[0028] Through the REE module, when the folding screen changes from the first display state to the second display state, screen information corresponding to the folding screen in the second display state is obtained.

[0029] The screen information is transmitted to the TEE module through the REE module.

[0030] Based on the first aspect, in a possible embodiment, in response to the second operation of the user on the UI of the CA, triggering display of the TUI of the TA in the TEE module on the screen corresponding to the second display state, and the TUI is adapted to the screen corresponding to the second display state, including:

[0031] Through the REE module, in response to a second operation of the user on the UI of the CA, a TUI display request is triggered.

[0032] The TUI display request is sent to the TEE module through the REE module, so as to call the TA corresponding to the CA in the TEE module.

[0033] Based on the first aspect, in a possible embodiment, in response to the second operation of the user on the UI of the CA, triggering display of the TUI of the TA in the TEE module on the screen corresponding to the second display state, and the TUI is adapted to the screen corresponding to the second display state, including:

[0034] The TUI display request is received through the TEE module.

[0035] Through the TEE module, the TUI of the TA in the TEE module is drawn according to the screen information, so that the TUI is adapted to the screen corresponding to the second display state.

[0036] The TUI is displayed on the screen corresponding to the second display state through the TEE module.

[0037] Based on the first aspect, in a possible embodiment, drawing the TUI of the TA in the TEE module according to the screen information so that the TUI is adapted to the screen corresponding to the second display state includes:

[0038] The coordinate transformation information is obtained according to the screen information through the TEE module.

[0039] The TUI is drawn according to the coordinate transformation information through the TEE module, and the size of the TUI is consistent with the size of the screen corresponding to the second display state, or the size of the TUI is smaller than the size of the screen corresponding to the second display state.

[0040] Based on the first aspect, in a possible embodiment, transmitting the screen information to the TEE module through the REE module includes:

[0041] The screen information is encrypted through the REE module to obtain encrypted screen information.

[0042] Through the REE module, it is verified whether the process for transmitting the encrypted screen information is a system service.

[0043] Through the REE module, when the process is a system service, a driver interface is called to transmit the encrypted screen information to the TEE module.

[0044] It can be seen that in the embodiment of the present invention, for an electronic device with a folding screen, when the display state of the folding screen where the application service is located changes, the screen information of the current folding screen (the latest screen information) can be transmitted to the adaptive display module on the TEE side through the REE module, so that the TEE side can obtain the screen information of the folding screen in real time, creating a feasible information transmission channel; the screen information is processed by the TEE module to realize the automatic adaptation of the TUI interface to the screen of the current folding screen, and the adaptive display of TUI is realized by using software algorithms (rather than GPU display), which can solve the pain points of folding screen users and greatly improve the user experience.

[0045] Based on the first aspect, in a possible embodiment, transmitting the screen information to the TEE module through the REE module includes:

[0046] The identity of the process used to transmit the screen information is verified according to the screen information through the TEE module.

[0047] Through the TEE module, when the verification is passed, the screen information is decrypted and obtained.

[0048] In a second aspect, an embodiment of the present invention provides an electronic device with a folding screen, wherein the folding screen includes at least two screens, and a client application CA runs on the electronic device, including:

[0049] A rich execution environment (REE) module is used to change the folding screen from a first display state to a second display state in response to a first operation of the user on the electronic device; wherein the first display state indicates that at least one of the at least two screens is in a display state, and the second display state indicates that at least one of the at least two screens is in a display state, and the screen corresponding to the first display state is different from the screen corresponding to the second display state; and is also used to display the user interface UI of the CA on the screen corresponding to the second display state.

[0050] A trusted execution environment TEE module is used to trigger the display of a trusted user interface TUI corresponding to the CA on the screen corresponding to the second display state in response to the second operation of the user on the UI of the CA, and the TUI is adapted to the screen corresponding to the second display state.

[0051] It can be seen that in the embodiment of the present invention, for an electronic device with a folding screen, when the display state of the folding screen where the application service is located changes, the embodiment of the present invention can realize the adaptive display of CA's UI on the folding screen through the REE module, and can also realize the adaptive display of TA's TUI on the folding screen through the TEE module, which can solve the pain points of folding screen users and greatly improve the user experience.

[0052] Based on the second aspect, in a possible embodiment, the first operation is an operation of unfolding the folding screen into a full-screen display.

[0053] The first display state indicates that one of the at least two screens is in a display state, and the second display state indicates that the at least two screens are in a display state.

[0054] Based on the second aspect, in a possible embodiment, the first operation is an operation of folding the folding screen into a single-screen display.

[0055] The first display state indicates that the at least two screens are in a display state, and the second display state indicates that one screen among the at least two screens is in a display state.

[0056] Based on the second aspect, in a possible embodiment, the first operation is an operation of flipping the electronic device to switch the folding screen from a first single-screen display to a second single-screen display.

[0057] The first display state indicates that a first screen among the at least two screens is in a display state, and the second display state indicates that a second screen among the at least two screens is in a display state.

[0058] Based on the second aspect, in a possible embodiment, the REE module is further used to: in response to a third operation of the user on the electronic device, determine that the folding screen is changed from the second display state to the third display state, the third display state indicating that at least one of the at least two screens is in a display state, and the screen corresponding to the third display state is different from the screen corresponding to the second display state; after the TUI in the screen corresponding to the third display state is exited, display the UI of the CA;

[0059] The TEE module is also used for: in response to a fourth operation of the user on the UI of the CA, triggering display of the TUI of the CA on the screen corresponding to the third display state, and the TUI is adapted to the screen corresponding to the third display state.

[0060] Based on the second aspect, in a possible embodiment, the fourth operation and the second operation are the same operation.

[0061] Based on the second aspect, in a possible embodiment, the screen corresponding to the third display state is the same as the screen corresponding to the first display state; or, the screen corresponding to the third display state is different from the screen corresponding to the first display state.

[0062] Based on the second aspect, in a possible embodiment, the adaptation of the TUI to the screen corresponding to the second display state means that: the TUI occupies the entire screen on the screen corresponding to the second display state.

[0063] Based on the second aspect, in a possible embodiment, the adaptation of the TUI to the screen corresponding to the second display state means that: the TUI occupies a part of the screen on the screen corresponding to the second display state.

[0064] Based on the second aspect, in a possible embodiment, the REE module is used to: display a UI of the CA in the REE module on a screen corresponding to the second display state.

[0065] The TEE module is used to: in response to a second operation of the user on the UI of the CA, trigger the display of the TUI of the TA in the TEE module on the screen corresponding to the second display state.

[0066] Based on the second aspect, in a possible embodiment, the REE module is specifically used for:

[0067] When the folding screen changes from the first display state to the second display state, screen information corresponding to the folding screen in the second display state is obtained; and the screen information is transmitted to the TEE module.

[0068] Based on the second aspect, in a possible embodiment, the REE module is specifically used to: in response to the second operation of the user on the UI of the CA, trigger a TUI display request, and send the TUI display request to the TEE module, so as to call the TA corresponding to the CA in the TEE module.

[0069] Based on the second aspect, in a possible embodiment, the TEE module is specifically used for:

[0070] Receive the TUI display request; draw the TUI of the TA in the TEE module according to the screen information to adapt the TUI to the screen corresponding to the second display state; display the TUI on the screen corresponding to the second display state.

[0071] Based on the second aspect, in a possible embodiment, the TEE module is specifically used for:

[0072] Coordinate transformation information is obtained according to the screen information. The TUI is drawn according to the coordinate transformation information, wherein the size of the TUI is consistent with the size of the screen corresponding to the second display state, or the size of the TUI is smaller than the size of the screen corresponding to the second display state.

[0073] Based on the second aspect, in a possible embodiment, the REE module is specifically used for:

[0074] The screen information is encrypted to obtain the encrypted screen information; whether the process used to transmit the encrypted screen information is a system service is verified; if the process is a system service, a driver interface is called to transmit the encrypted screen information to the TEE module.

[0075] Based on the second aspect, in a possible embodiment, the REE module is specifically used for:

[0076] The identity of the process used to transmit the screen information is verified according to the screen information; if the verification is successful, the screen information is decrypted to obtain the screen information.

[0077] Based on the second aspect, in a possible embodiment, in order to enable the trusted user interface (TUI) of the TA corresponding to the CA to automatically adapt to the screen changes of the folding screen, a screen state perception module is also deployed in the user state of REE, and an adaptive display module is also deployed in the user state of TEE.

[0078] The screen status perception module can be a process (daemon) of a system service. In an embodiment of the present invention, the screen status perception module can be used to monitor the display status of the folding screen. For example, the screen status perception module can monitor broadcasts in the Android® operating system. When the display status of the folding screen changes, the Android® operating system broadcasts the change event of the display status of the folding screen. The screen status perception module obtains screen information through monitoring, and the screen information is used to indicate the current display status of the folding screen. The screen information may, for example, include the display status information of different screens in the current folding screen. Furthermore, the screen information may also include information such as the size and resolution of different screens in the folding screen.

[0079] The screen state perception module can also send screen information to the adaptive display module in the TEE. For example, the screen state perception module can call the TUI-switch function or the TUI-agent function, and use the global task function in the TEE to send the screen information to the adaptive display module.

[0080] In one implementation, the screen state perception module may include a code program of a monitoring function and a program code of a TEE transmission function. The code program of the monitoring function may be used to obtain the screen information of the current folding screen by monitoring system broadcasts (e.g., Android system broadcasts). The program code of the TEE transmission function may be used to send the screen information to the TEE, for example, the screen information may be transmitted to the TEE side through a TUI agent.

[0081] In one implementation, the screen state perception module may include a code program of a monitoring function and a program code of a TEE transmission function. The code program of the monitoring function may be used to obtain the screen information of the current folding screen by monitoring system broadcasts (e.g., Android system broadcasts), and may also be used to send the screen information to the GPU so that the GPU can realize adaptive display of the UI interface. The program code of the TEE transmission function may be used to send the screen information to the TEE, for example, the screen information may be transmitted to the TEE side through the TUI agent.

[0082] The adaptive display module is used to adaptively display the TA's TUI to the current folding screen according to the screen information, that is, to adapt the TUI interface to the screen in the display state of the current folding screen. Specifically, the received screen information is dynamically saved in the adaptive display module, that is, when new screen information is received, the old screen information is replaced locally with the new screen information. The adaptive display module can perform coordinate conversion processing based on the screen information to obtain processed screen information. Based on the processed screen information, the size, resolution, etc. of the called TA's TUI are processed to achieve the adaptive display of TUI on the folding screen.

[0083] Based on the second aspect, in a possible embodiment, in order to ensure the security of screen information transmission, not only a screen status perception module but also an identity authentication module is deployed in the user state of REE; not only an adaptive display module but also a session key verification module is deployed in the user state of TEE.

[0084] The screen status sensing module may be a system service process (daemon). In an embodiment of the present invention, the screen status sensing module may be used to monitor the display status of the folding screen.

[0085] Furthermore, the screen information also includes a challenge value (or a random number), and the screen state sensing module encrypts the screen information (for example, encrypts it in a symmetric encryption manner) to obtain the encrypted screen information.

[0086] Alternatively, the screen state sensing module may encrypt the screen information (for example, by encrypting it in a symmetric encryption manner). Then, the screen state sensing module may encapsulate the encrypted screen information and the challenge value (or random number) into a data packet.

[0087] The screen status sensing module may also send the encrypted screen information or the data packet to the identity authentication module in the REE.

[0088] The identity authentication module is used to perform preliminary authentication on the identity of the information transmitter (such as the identity of the screen status perception module in this article), and only allows the system service to call the information transmission interface. Specifically, the identity authentication module can obtain the identity information of the information transmitter through the process information (process) to determine whether the information transmitter is a system service. In this way, when the information transmitter is the screen status perception module designed in the embodiment of this article, since the identity authentication module can determine that the screen status perception module is a system service, the identity authentication module will allow the screen status perception module to call the information transmission interface (call the TUI-switch function or the TUI-agent function) and send the above-mentioned encrypted screen information or the above-mentioned data packet to the TEE. When the information transmitter is a function code forged by a malicious attacker, the identity authentication module confirms that the information transmitter is not a system service, and will refuse the information transmitter to call the relevant interface to avoid unsafe information from being transmitted to the TEE side, thereby ensuring the information security of the REE side and the TEE side.

[0089] The session key verification module and the adaptive display module can be deployed independently in the user state of TEE, or the functions can be integrated and deployed in the user state of TEE. For convenience, the following will take the example that the session key verification module and the adaptive display module can be deployed independently in the user state of TEE to describe the solution.

[0090] The session key verification module is used to decrypt the screen information and verify the identity information of the information transmitter based on the challenge value (random number) carried by it. For example, when the challenge value (random number) is verified correctly, the identity of the information transmitter is confirmed to be legal, that is, the information transmitter is confirmed to be the screen status perception module designed in the embodiment of the present application, rather than a function code disguised by a malicious attacker. This verification can further improve the information security on the TEE side, for example, to ensure that the screen information will not be tampered with after coming out of the aforementioned identity authentication module.

[0091] The session key verification module further transmits the decrypted screen information to the adaptive display module.

[0092] The adaptive display module is used to adaptively display TA's TUI to the current folding screen according to the screen information, that is, to adapt the TUI interface to the screen in the display state in the current folding screen.

[0093] In a third aspect, an embodiment of the present invention provides an electronic device, which may include: one or more processors, a memory and a folding screen; the folding screen may include at least two screens; the one or more processors, the memory, and the folding screen may be connected via one or more communication buses; one or more computer programs are stored in the above-mentioned memory, and one or more processors are used to execute the one or more computer programs; the one or more computer programs include instructions, which are used to execute the method steps described in the possible implementation example of the first aspect above.

[0094] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor executes the method described in any embodiment of the first aspect.

[0095] In a fifth aspect, an embodiment of the present invention provides a computer program product, which, when running on a computer, is executed to implement the method described in any embodiment of the first aspect.

[0096] It can be seen that in the embodiment of the present invention, for an electronic device with a folding screen, when the display state of the folding screen where the application service is located changes, the screen information of the current folding screen (the latest screen information) can be transmitted to the TEE side through REE, so that the TEE side can obtain the screen information of the folding screen in real time, creating a feasible information transmission channel; the screen information is processed by the TEE side to realize the automatic adaptation of the TUI interface to the screen of the current folding screen, and the adaptive display of TUI is realized by using software algorithms (rather than GPU display), which can solve the pain points of folding screen users and greatly improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0097] Figure 1 is an example diagram of a folding screen provided by an embodiment of the present invention;

[0098] Figure 2 is an example diagram of a folding screen provided by an embodiment of the present invention;

[0099] Figure 3 is an example diagram of a folding screen provided by an embodiment of the present invention;

[0100] Figure 4A-4C is an example diagram of three display states of a folding screen provided by an embodiment of the present invention;

[0101] Figure 5 is an architecture diagram of an electronic device provided by an embodiment of the present invention;

[0102] Figure 6 is an architecture diagram of an electronic device provided by an embodiment of the present invention;

[0103] Figure 7 It is a schematic diagram of a system architecture provided by an embodiment of the present invention;

[0104] Figure 8 It is a schematic diagram of an application scenario in which TUI in an existing solution cannot be adapted to a folding screen;

[0105] Fig. 9 is a schematic diagram of an application scenario in which a TUI is adapted to a folding screen provided by an embodiment of the present invention;

[0106] Fig.10 is a schematic diagram of an application scenario in which a TUI is adapted to a folding screen provided by an embodiment of the present invention;

[0107] Fig.11 is a schematic diagram of an application scenario in which a TUI is adapted to a folding screen provided by an embodiment of the present invention;

[0108] Fig.12 is a schematic diagram of an application scenario in which a TUI is adapted to a folding screen provided by an embodiment of the present invention;

[0109] Fig.13 This is a schematic diagram of an application scenario of mobile banking transfer provided by an embodiment of the present invention;

[0110] Fig.14 This is a schematic diagram of an application scenario of mobile banking transfer provided by an embodiment of the present invention;

[0111] Fig.15 This is a schematic diagram of an application scenario of mobile banking transfer provided by an embodiment of the present invention;

[0112] Fig.16 is a flow chart of a TUI display method provided by an embodiment of the present invention;

[0113] Fig.17A An example diagram of a physical coordinate system used in different display states of a folding screen provided by an embodiment of the present invention;

[0114] Fig. 17B is an example diagram of a drawing coordinate system used in different display states of a folding screen provided by an embodiment of the present invention;

[0115] Fig.18 is a flow chart of a TUI display method provided by an embodiment of the present invention;

[0116] Fig.19 is a flowchart of a TUI display provided by an embodiment of the present invention;

[0117] Fig. 20 is a flow chart of a TUI display method provided by an embodiment of the present invention;

[0118] Fig.21 is a flow chart of a TUI display method provided by an embodiment of the present invention;

[0119] Fig. 22 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention;

[0120] Fig.23 It is a schematic diagram of the structure of a device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0121] The technical solutions in the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0122] The terms "user interface (UI)" and / or "trusted user interface (TUI)" in the specification, claims and drawings of the present invention may be a graphical interface for interaction and information exchange between an application or operating system and a user. It may be a window, dialog box, display area, etc. visible to the user. The user interface of an application may be, for example, source code written in a specific computer language such as Java or extensible markup language (XML). The interface source code is parsed and rendered on a terminal device and finally presented as user-visible content, such as pictures, text, buttons, etc. A view is a visual component of a user interface. A view is also called a control or widget. Typical views include buttons, text fields, progress bars, keyboards, pictures, text, etc.

[0123] The display method of the trusted user interface provided in the embodiment of the present invention can be applied to an electronic device with a folding screen. The electronic device in the embodiment of the present invention can be a mobile phone, a tablet computer, a desktop, a laptop, a handheld computer, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a cellular phone, a personal digital assistant (PDA), an augmented reality (AR)\virtual reality (VR) device, etc., which includes a folding screen. The embodiment of the present invention does not impose any special restrictions on the specific form of the device.

[0124] The folding screen is first described below.

[0125] An electronic device with a folding screen includes at least two screens, and the sizes of each screen may be different or the same. In this article, "screen" may also be referred to as a display screen, a screen, a display panel, etc. In this article, a screen may refer to an independent and complete screen (or an independent display unit, such as one that can be controlled by a processor as a whole), or may refer to a display area of ​​a part of a complete screen. The electronic device may control the display state of different screens separately. Based on the control, the display state of the screen may specifically be a state in which the screen is turned on and displayed (such as a bright screen state), or a state in which the screen is turned off and displayed (such as a black screen state). Different screens may be movably connected based on a connecting mechanism (e.g., a hinge), and in some implementations, the outside of the connecting mechanism is also covered with a display screen (which may be referred to as a connecting screen or a folding edge). In this way, different screens of the folding screen may be flattened and unfolded into a full screen for display based on external force (such as a user unfolding it by hand). It may also be folded into a single screen for display based on external force (such as a user folding it by hand). For example, when unfolded to full screen, it can be displayed as an 8-inch full screen, and when folded into a single screen, it can be displayed as a 6.6-inch screen or a 6.38-inch screen.

[0126] Generally, there are two folding schemes for folding screens: one is outward folding and the other is inward folding. Outward folding is to fold the screen of the folding screen in half, with one screen on the front and one screen on the back. Inward folding is to close the screen like a notebook, and then design a screen on the outside to display after folding.

[0127] Exemplarily, in some embodiments of the present invention, the folding screen may be a flexible folding screen. The flexible folding screen includes a folding edge made of a flexible material. Part or all of the flexible folding screen is made of a flexible material. For example, only the foldable part of the flexible folding screen (called the connecting screen or folding edge) is made of a flexible material, and the other parts are made of a rigid material; or, the flexible folding screen is made entirely of a flexible material. The folding screen can be folded along the folding edge to form multiple (two or more) screens.

[0128] In various embodiments of the present invention, the folding screen includes at least two screens, and the adjacent screens can rotate around the folding axis to achieve the unfolding or folding of the folding screen. In other words, the screen state of the folding screen may include: unfolded state and folded state.

[0129] The unfolded state means that the folding screen is fully unfolded (i.e., the full screen), and the angle between any two screens in the folding screen is 180°. When the full screen is used for display, the display state of the folding screen is that each screen in the full screen is in the display state.

[0130] The folded state means that the foldable screen is completely folded (i.e., folded into a single screen), and the angle between any two screens in the foldable screen is 0° or 360°. When a single screen in the foldable screen is used for display, the display state of the foldable screen is that the single screen is in the display state (or turned on state, or bright screen state), while other single screens may be in the turned off state (or non-display state, or black screen state).

[0131] See also Figure 1 ,by Figure 1 Take the folding screen shown in the figure as an example. The folding screen includes two independent single screens and a folding edge made of flexible material. The folding screen is folded outward along the folding edge to form Figure 1 The A screen 101 (single screen) and the B screen 102 (single screen) shown in (a) of FIG. 1 are shown in FIG. 2 . The angle α between the A screen 101 and the B screen 102 is 360°. When the user gradually unfolds the folding screen, the angle α between the A screen 101 and the B screen 102 gradually approaches 180° from 360°. Figure 1 As shown in (b) in FIG. 1 , it can be understood that when the angle α between the A screen 101 and the B screen 102 is 180°, it is a full screen.

[0132] Above Figure 1 The folding screen of the electronic device is folded longitudinally to realize the folding of the folding screen. Of course, in a possible implementation, the folding screen of the electronic device can also be folded transversely to realize the folding of the folding screen, which is not limited here.

[0133] See also Figure 2 ,by Figure 2Taking the folding screen shown in the figure as an example, the folding screen includes two independent single screens and a folding edge made of a flexible material. The folding screen is folded inward along the folding edge. Figure 2 (a) is the full screen. When the user gradually folds the foldable screen, the Figure 2 (b) Figure 2 (c) and Figure 2 The process shown in (d) in the figure is to fold the folding screen inward along the folding edge. It can be understood that when the screen is full screen, the angle between screen A and screen B is 180°. During the folding process, the angle between screen A and screen B gradually decreases. When the angle between screen A and screen B is 0°, screen A and screen B are completely folded together. At this time, the rear screen C can be used for display, that is, screen C is a single screen.

[0134] Above Figure 2 The folding screen of the electronic device is folded horizontally to achieve the folding of the folding screen. Of course, in a possible implementation, the folding screen of the electronic device can also be folded vertically to achieve the folding of the folding screen, which is not limited here.

[0135] See also Figure 3 ,by Figure 3 Taking the folding screen shown in the figure as an example, the multi-screen folding screen can also include three (or more) independent screens, with folding edges made of flexible materials between adjacent screens. These screens can be connected in sequence through folding axes. Each screen can rotate around the folding axis connected to it. These screens can be unfolded to full size or folded into multiple single screens. Figure 3 As shown, the folding screen can be folded outward along the folding edge. After the folding screen is folded along the folding edge, a Figure 3 Shown are an A screen 201 (single screen), a B screen 202 (single screen) and a C screen 203 (single screen).

[0136] It should be noted that the above Figure 1-Figure 3 The folding screen shown is only used to exemplify the solution of the present invention and is not limiting.

[0137] The following continues to describe the display state of the folding screen involved in the embodiment of the present invention, and the change of the display state.

[0138] See also Figure 4A-4C , Figure 4A-4C This is a schematic diagram of three display states of an exemplary folding screen. The folding screen may include two independent single screens (screen A and screen B) and a folding edge (or connecting screen) made of flexible material. Among them, the screen size of screen A is greater than or equal to the screen size of screen B. For convenience, screen A can be defined as the main screen of the folding screen, and screen B can be defined as the secondary screen of the folding screen. When screen A and screen B are flatly unfolded, that is, the angle between the two is 180°, the unity formed by screen A, screen B and the folding edge (or connecting screen) between the two can be called a full screen.

[0139] Figure 4A This is a schematic diagram of the unfolded state (full screen). When the folding screen is in the working state (display state), it means that screen A, screen B and the connecting screen between the two in the full screen are all in the display state. In this way, any area in the full screen can be used to display business content, and the user can perform related operations on the business content displayed in the full screen, including but not limited to click, slide, touch, approach, press and other operations.

[0140] Figure 4B This is a schematic diagram of a folding state (single screen). For example, in one implementation, when the folding screen is in the folded state, the user holds the folding screen with one hand, so that the user's thumb is on the main screen side, and the other fingers are on the secondary screen side of the folding screen, that is, the main screen faces the user. At this time, when the folding screen is in the working state (display state), the main screen can be set to the display state, which can be used to display business content, and the secondary screen is not in the display state (such as not displaying business content or a black screen).

[0141] Figure 4C This is a schematic diagram of another folding state (single screen). For example, in one implementation, when the folding screen is in the folded state, the user holds the folding screen with one hand, so that the user's thumb is on the secondary screen side, and the other fingers are on the main screen side of the folding screen, that is, the secondary screen faces the user. At this time, when the folding screen is in the working state (display state), the secondary screen can be set to the display state, which can be used to display business content, and the main screen is not in the display state (such as not displaying business content or a black screen).

[0142] In an embodiment of the present invention, the folding screen of the electronic device includes at least two independent single screens, and the user can perform a certain operation (or first operation) on the electronic device with the folding screen, so as to change the working state (display state) of the folding screen from the first display state to the second display state. The first display state indicates that at least one of the at least two screens is in the display state, and the second display state indicates that at least one of the at least two screens of the folding screen is in the display state, and the screen corresponding to the first display state is different from the screen corresponding to the second display state.

[0143] Exemplarily, the first display state may be the above Figure 4A-4C A display state of a folding screen as described in. The second display state may be the above Figure 4A-4C Another display state of a folding screen described in .

[0144] For example, the first display state may be Figure 4B The display state of the folding screen shown (i.e., the main screen display) and the second display state can be Figure 4AThe display state of the folding screen shown (ie, full-screen display) is such that the user can change the folding screen from the first display state to the second display state by performing an unfolding operation (first operation) on the folding screen.

[0145] For another example, the first display state may be Figure 4C The display state of the folding screen shown (i.e., the secondary screen display), the second display state can be Figure 4A The display state of the folding screen shown (ie, full-screen display) is such that the user can change the folding screen from the first display state to the second display state by performing an unfolding operation (first operation) on the folding screen.

[0146] For another example, the first display state may be Figure 4A The display state of the folding screen shown (i.e. full screen display) and the second display state may be Figure 4B The display state of the folding screen shown (i.e., the main screen display) can be changed from the first display state to the second display state by the user performing a folding operation (first operation) on the folding screen.

[0147] For another example, the first display state may be Figure 4B The display state of the folding screen shown (i.e., the main screen display) and the second display state can be Figure 4C The display state of the folding screen shown (i.e., the secondary screen display) can be changed from the first display state to the second display state by the user by performing a flipping operation (first operation) on the electronic device.

[0148] For another example, the first display state may be Figure 4B The display state of the folding screen shown (i.e., the main screen display) and the second display state can be Figure 4C The display state of the folding screen shown (i.e., the secondary screen display) can be changed from the first display state to the second display state by the user by performing a flipping operation (first operation) on the electronic device.

[0149] For another example, the first display state may be Figure 4C The display state of the folding screen shown (i.e., the secondary screen display), the second display state can be Figure 4B The display state of the folding screen shown (i.e., the main screen display) is such that the user can change the folding screen from the first display state to the second display state by performing a flipping operation (a first operation) on the electronic device.

[0150] It should be noted that in this article, the so-called "single screen", "independent single screen" and "screen" should be understood to have the same meaning. These terms are used in this article for the convenience of description. They can refer to an independent and complete screen (or an independent display unit, such as one that can be controlled by a processor as a whole), or a part of the display area of ​​a complete screen. For example Figure 4A In the display, the main screen (screen A) and the sub-screen (screen B) can be a complete screen respectively, or two display areas in the full screen (in this case, the full screen can be considered as an independent and complete screen).

[0151] It should also be noted that the folding screens described in the above embodiments and related drawings are only used to explain the technical solutions of this application and are not intended to be limiting. There are no limitations on the shape, appearance, material, number of screens, connection method between screens, and folding method of each screen of the folding screen, and there are no limitations on the deployment location and number of other hardware that may exist in the electronic device, such as cameras, sensors, buzzers, and other components.

[0152] A system architecture of an embodiment of the present invention is described below.

[0153] Figure 5 1 shows an architecture diagram of an electronic device 100 provided by an embodiment of the present invention, such as Figure 5 As shown, the electronic device 100 includes a hardware platform, and two mutually isolated operating environments running on the hardware platform, namely a rich execution environment (REE) and a trusted execution environment (TEE), and the two operating environments have independent hardware resources and operating systems. In this article, REE and TEE may also be referred to as REE module and TEE module, respectively. Through hardware isolation technology, such as the ARM® TrustZone mechanism, the isolation of the hardware resources of REE and TEE can be achieved, and the isolation between the operating systems corresponding to REE and TEE, and between applications can be achieved through virtualization technology. In this way, the software and hardware resources accessible to TEE are separated from REE, and TEE has very strict restrictions on the data and functions accessible to the application program, so that its security level meets specific security requirements, so TEE can be considered a secure execution environment. REE is an operating environment outside of TEE, and compared to TEE, it can also be called a non-secure execution environment.

[0154] The hardware platform of the electronic device 100 includes, for example, public peripherals and trusted peripherals, and the trusted peripherals include secure elements (SE) that can only be controlled and accessed by TEE, such as secure memory, secure clock, trusted keyboard, etc. Public peripherals are devices that can be controlled and accessed by the operating system in REE.

[0155] An application running in TEE is called a trusted application (TA). There can be one or more TAs. Figure 5Only two TAs are used as examples. The interface of a TA can be called a trusted user interface (TUI). The application running in the REE is called a client application (CA). There can be one or more CAs. Figure 5 Only two CAs are used as examples). The interface of CA can be called User Interface (UI). For example, CA can be WeChat Pay, Alipay, Apple Pay App, Samsung Pay App, Huawei Pay App, bank client, mobile shield app, electronic ID card, mobile POS or other application software involving the input of sensitive information such as account number and password; TA is the security application corresponding to CA, which is used to input sensitive information involved in CA.

[0156] For definitions of terms such as REE, TEE, CA, TA, etc. involved in all embodiments of the present invention, please refer to the TEE-related standards proposed by the Global Platform® (GP).

[0157] The TA running in the TEE can provide security-related functions or services for the CA in the REE or other TAs in the TEE. The trusted operating system running in the TEE can provide the TA with the TEE internal interface, and the TA obtains access rights to security resources and services through the TEE internal interface. These security resources and services include but are not limited to: key injection and management, encryption, secure storage, secure clock, trusted user interface (TUI) and trusted keyboard, etc.

[0158] The CA running in REE can use the external interface provided by TEE to request the security services provided by TA in TEE. The operating system running in REE (such as Android®, iOS®, Windows Phone and other terminal operating systems) can provide richer features than the trusted operating system in TEE and can accept various types of applications, but its security is also lower than that of the trusted operating system.

[0159] For example, in scenarios such as mobile payment and online bank transfer, if the input and display of user sensitive information is involved, the CA in REE can call the TUI and trusted keyboard services on the TEE side through the external interface provided by TEE to prevent malicious programs on the REE side from eavesdropping and stealing user sensitive information.

[0160] The architecture based on Linux system (such as Android® operating system) can also be divided into user mode and kernel mode. The kernel is essentially a kind of software - it controls the hardware resources of the computer and provides an environment for the upper-level applications to run. User mode is the activity space of the upper-level applications. The execution of applications must rely on the resources provided by the kernel, including CPU resources, storage resources, I / O resources, etc. In order for the upper-level applications to access these resources, the kernel must provide an access interface for the upper-level applications, namely the system call.

[0161] It should be understood that CA runs in the user state of REE, and TA runs in the user state of REE. A driver module is deployed in the kernel state of REE (for example, including TZdriver function, which provides a driver interface for REE to access TEE); a driver module is also deployed in the kernel state of TEE; the driver modules in REE and TEE can access the corresponding hardware devices, for example, TA can realize the display of CA's UI on the display screen by calling GPU. The driver module of REE may also include TUI conversion (TUI-switch) function or TUI agent (TUI-agent) function. In addition, REE control module can also be deployed in REE, for example, REE control module may include the function of process (daemon), not shown in the figure), TEE control module (not shown in the figure) can also be deployed in TEE, CA can access TA through REE control module and TEE control module to achieve corresponding security operations. For example, the REE control module can call the driver module on the REE side to drive the hardware device to exit the non-secure working mode according to the CA's TUI access request (or TUI display request); after the hardware device exits the non-secure working mode, the TEE control module can call the driver module on the TEE side to drive the hardware device to switch the working mode according to the message sent by the REE control module to achieve hardware isolation from the REE, and then call the corresponding TA to achieve CA's access, signature, confirmation, etc. to the TA and display the TA's TUI on the display screen. The above-mentioned specific functions of the REE driver module, TEE driver module, REE control module, TEE control module, etc. can all be implemented by the processor in the electronic device.

[0162] In an embodiment of the present invention, in order to enable the trusted user interface (TUI) of the TA corresponding to the CA to automatically adapt to the screen changes of the folding screen, a screen state perception module is also deployed in the user state of the REE, and an adaptive display module is also deployed in the user state of the TEE.

[0163] The screen status perception module can be a process (daemon) of a system service. In an embodiment of the present invention, the screen status perception module can be used to monitor the display status of the folding screen. For example, the screen status perception module can monitor broadcasts in the Android® operating system. When the display status of the folding screen changes, the Android® operating system broadcasts the change event of the display status of the folding screen. The screen status perception module obtains screen information through monitoring, and the screen information is used to indicate the current display status of the folding screen. The screen information may, for example, include the display status information of different screens in the current folding screen. Furthermore, the screen information may also include information such as the size and resolution of different screens in the folding screen.

[0164] The screen state perception module can also send screen information to the adaptive display module in the TEE. For example, the screen state perception module can call the TUI-switch function or the TUI-agent function, and use the global task function in the TEE to send the screen information to the adaptive display module.

[0165] In one implementation, the screen state perception module may include a code program of a monitoring function and a program code of a TEE transmission function. The code program of the monitoring function may be used to obtain the screen information of the current folding screen by monitoring system broadcasts (e.g., Android system broadcasts). The program code of the TEE transmission function may be used to send the screen information to the TEE, for example, the screen information may be transmitted to the TEE side through a TUI agent.

[0166] In one implementation, the screen state perception module may include a code program of a monitoring function and a program code of a TEE transmission function. The code program of the monitoring function may be used to obtain the screen information of the current folding screen by monitoring system broadcasts (e.g., Android system broadcasts), and may also be used to send the screen information to the GPU so that the GPU can realize adaptive display of the UI interface. The program code of the TEE transmission function may be used to send the screen information to the TEE, for example, the screen information may be transmitted to the TEE side through the TUI agent.

[0167] The adaptive display module is used to adaptively display the TA's TUI to the current folding screen according to the screen information, that is, to adapt the TUI interface to the screen in the display state of the current folding screen. Specifically, the received screen information is dynamically saved in the adaptive display module, that is, when new screen information is received, the old screen information is replaced locally with the new screen information. The adaptive display module can perform coordinate conversion processing based on the screen information to obtain processed screen information. Based on the processed screen information, the size, resolution, etc. of the called TA's TUI are processed to achieve the adaptive display of TUI on the folding screen.

[0168] In this article, an interface is also called an application programming interface (API), which is an encapsulation and abstract expression of a specific function implemented by a computer program code. An application can implement a specific function by calling the interface. A service (such as a system service) is an application component that can be executed in the background without providing a user interface. A service can be started by other applications or application components.

[0169] It can be seen that in the embodiment of the present invention, for an electronic device with a folding screen, when the display state of the folding screen where the application business is located changes, the screen information of the current folding screen (the latest screen information) can be transmitted to the TEE side through the screen state perception module, so that the TEE side can obtain the screen information of the folding screen in real time, creating a feasible information transmission channel; the screen information is processed by the adaptive display module to realize the automatic adaptation of the TUI interface to the screen of the current folding screen, which can solve the pain points of folding screen users and greatly improve the user experience.

[0170] Figure 6 2 shows an architecture diagram of another electronic device 200 provided by an embodiment of the present invention, such as Figure 6 As shown, the electronic device 200 also includes a hardware platform, and two isolated operating environments running on the hardware platform, namely REE and TEE, and the two operating environments have independent hardware resources and operating systems. REE and TEE may also be referred to as REE module and TEE module respectively in this article. Through hardware isolation technology, the isolation of hardware resources of REE and TEE can be achieved, and the isolation between the operating systems corresponding to REE and TEE, and between applications can be achieved.

[0171] Figure 6 The electronic device 200 and Figure 5 The difference in architecture of the electronic device 100 shown is mainly that, in order to ensure the security of screen information transmission, Figure 6 In the architecture of the electronic device 200 shown, not only a screen status perception module but also an identity authentication module is deployed in the user state of REE; not only an adaptive display module but also a session-key verification module is deployed in the user state of TEE.

[0172] The screen status perception module can be a process (daemon) of a system service. In an embodiment of the present invention, the screen status perception module can be used to monitor the display status of the folding screen. For example, the screen status perception module can monitor broadcasts in the Android® operating system. When the display status of the folding screen changes, the Android® operating system broadcasts the change event of the display status of the folding screen. The screen status perception module obtains screen information through monitoring, and the screen information is used to indicate the current display status of the folding screen. The screen information may, for example, include the display status information of different screens in the current folding screen. Furthermore, the screen information may also include information such as the size and resolution of different screens in the folding screen.

[0173] Furthermore, the screen information also includes a challenge value (or a random number), and the screen state sensing module encrypts the screen information (for example, encrypts it in a symmetric encryption manner) to obtain the encrypted screen information.

[0174] Alternatively, the screen state sensing module may encrypt the screen information (for example, by encrypting it in a symmetric encryption manner). Then, the screen state sensing module may encapsulate the encrypted screen information and the challenge value (or random number) into a data packet.

[0175] The screen status sensing module may also send the encrypted screen information or the data packet to the identity authentication module in the REE.

[0176] The identity authentication module is used to perform preliminary authentication on the identity of the information transmitter (such as the identity of the screen status perception module in this article), and only allows the system service to call the information transmission interface. Specifically, the identity authentication module can obtain the identity information of the information transmitter through the process information (process) to determine whether the information transmitter is a system service. In this way, when the information transmitter is the screen status perception module designed in the embodiment of this article, since the identity authentication module can determine that the screen status perception module is a system service, the identity authentication module will allow the screen status perception module to call the information transmission interface (call the TUI-switch function or the TUI-agent function) and send the above-mentioned encrypted screen information or the above-mentioned data packet to the TEE. When the information transmitter is a function code forged by a malicious attacker, the identity authentication module confirms that the information transmitter is not a system service, and will refuse the information transmitter to call the relevant interface to avoid unsafe information from being transmitted to the TEE side, thereby ensuring the information security of the REE side and the TEE side.

[0177] In the embodiment of the present invention, the session key verification module and the adaptive display module can be independently deployed in the user state of the TEE, or the functions can be integrated and deployed in the user state of the TEE. For convenience, the following will take the example that the session key verification module and the adaptive display module can be independently deployed in the user state of the TEE as an example to describe the solution.

[0178] After receiving the encrypted screen information or data packet transmitted by REE, the global task function in TEE sends the encrypted screen information or data packet to the session key verification module.

[0179] The session key verification module is used to decrypt the screen information and verify the identity information of the information transmitter based on the challenge value (random number) carried by it. For example, when the challenge value (random number) is verified correctly, the identity of the information transmitter is confirmed to be legal, that is, the information transmitter is confirmed to be the screen status perception module designed in the embodiment of the present application, rather than a function code disguised by a malicious attacker. This verification can further improve the information security on the TEE side, for example, to ensure that the screen information will not be tampered with after coming out of the aforementioned identity authentication module.

[0180] The session key verification module further transmits the decrypted screen information to the adaptive display module.

[0181] The adaptive display module is used to adaptively display the TA's TUI to the current folding screen according to the screen information, that is, to adapt the TUI interface to the screen in the display state of the current folding screen. Specifically, the received screen information is dynamically saved in the adaptive display module, that is, when new screen information is received, the old screen information is replaced locally with the new screen information. The adaptive display module can perform coordinate conversion processing based on the screen information to obtain processed screen information. Based on the processed screen information, the size, resolution, etc. of the called TA's TUI are processed to achieve the adaptive display of TUI on the folding screen.

[0182] It can be seen that in the embodiment of the present invention, for an electronic device with a folding screen, when the display state of the folding screen where the application service is located changes, the screen information of the current folding screen (the latest screen information) can be encrypted and transmitted to the adaptive display module on the TEE side through the screen state perception module, so that the TEE side can obtain the screen information of the folding screen in real time. And in this process, the identity authentication module is used to perform preliminary identity authentication on the REE side, and the session key verification module is used to perform further identity authentication on the TEE side, creating a safe and reliable information transmission channel to ensure information security on the REE side and the TEE side. By processing the screen information through the adaptive display module, the TUI interface can be automatically adapted to the current folding screen, which can solve the pain points of folding screen users and greatly improve the user experience.

[0183] See also Figure 7 , Figure 7 : is a schematic diagram of a system architecture provided by an embodiment of the present invention, the system architecture includes a bank server 231, a CFCA system server 232, an enterprise server 233 and an electronic device 211 with a folding screen. The electronic device 211 can communicate with the bank server 231, the CFCA system server 232 and the enterprise server 233, for example, wirelessly. The bank server 231, the CFCA system server 232 and the enterprise server 233 can also communicate with each other, for example, by wired or wireless communication. CFCA stands for China Financial Certification Authority.

[0184] For example, the bank server 231 may include a mobile banking management system, a CFCA application front-end, a bank RA system, etc.

[0185] For example, the CFCA system server 232 may include CFCA TSM, RA system, CA system, mobile certificate application platform, etc. Among them, the RA system is responsible for the information entry, review and certificate issuance of certificate applicants. CFCATSM is a trusted service manager, which is used to implement SE security domain management, including the distribution of applets, cos and U-shield certificates, etc. The CA system is responsible for issuing and maintaining certificates.

[0186] For example, the enterprise server 233 may include an eSE TSM for implementing a security domain (creation of SSD) within the terminal SE and distributing SE management keys to the CFCA TSM.

[0187] For example, an electronic device may include a REE and a TEE. The REE may run a CA, display a UI, and interact with a user, and may also run a CFCA plug-in. The TEE may run a TA (e.g., a CFCA TA), display a TUI, and interact with a user. In addition, the REE may also include a screen state perception module, an identity authentication module, etc. provided in an embodiment of the present invention, and the TEE may also include an adaptive display module, a session key verification module, etc. provided in an embodiment of the present invention. The electronic device may include a security processing unit (SE) module. Among them:

[0188] The CFCA plug-in is the initiating module of the business process, which receives the U-Shield certificate management request initiated by the user (through the bank apk or mobile wallet, etc.) and executes the business logic.

[0189] CFCA TA can be a business processing module on the mobile phone U shield TEE side, used to call fingerprint TA (FP TA) and TUI according to business logic, and communicate with the security processing unit (SE) module. FP TA is used to implement the user identity authentication function in fingerprint mode, and is called by CFCA TA to obtain the identity authentication result.

[0190] TUI is used to realize the confirmation of user transaction information and the display of identity authentication interface in a secure environment.

[0191] The SE module implements security domain creation, applet management, user identity authentication, U-shield certificate storage, and transaction information signing functions. For example, applets and U-shield certificates are placed in SE for protection. SE can include ISD and SSD. ISD is the primary security domain, the primary security domain within SE, production line configuration, and is responsible for SE basic management. SSD is the secondary security domain, the secondary domain within SE, used to install applets.

[0192] In an embodiment of the present invention, the display of the trusted user interface TUI on the display screen can be triggered based on the user's operation (second operation). After mobile payment, mobile banking, online banking transfer or any other service that requires calling TA is initiated, CA runs on the operating system of REE, and the interface of CA's UI is adaptively displayed on the folding screen. It can be understood that the interface of the UI is a graphical interface for interaction and information exchange between the CA and the user, so the user can perform related operations on the business content presented by the UI, including but not limited to click, slide, touch, approach, press, information input, information upload, information download and other operations.

[0193] When the business processing link reaches the interface of calling TA's TUI, the user needs to perform an operation on CA's UI. This operation can also be called the second operation in this article. The second operation is not limited to click, slide, touch, approach, press, information input, information upload, information download and other operations. Based on the user's second operation on CA's UI, the TUI is called and displayed on the folding screen. Based on the above-mentioned design of the present invention Figure 5 or Figure 6 According to the architecture shown, the TUI can be adapted to the current folding screen.

[0194] See also Figure 8 , Figure 8 An application scenario in which TUI in the existing solution cannot be adapted to the folding screen is given.

[0195] exist Figure 8 In the scenario (1), the user logs in to the mobile banking application (i.e., CA) on the REE side of the electronic device. The UI of the mobile banking application (i.e., the transfer interface in the figure) is presented on the current folding screen. In the figure, the current folding screen is in a single-screen (such as the main screen) display state. When the user enters the amount and clicks "Next Page" in the transfer interface, the TUI of the TA required by the mobile banking application is called out and presented on the single screen.

[0196] exist Figure 8 In the scenario (2), the user logs into the mobile banking application on the REE side on the electronic device, and the UI of the mobile banking application (i.e., the transfer interface in the figure) is presented on the current folding screen. In the figure, the current folding screen is in a single screen (e.g., main screen) display state. At this time, for convenience, the user unfolds the folding screen from the single screen to the full screen. The transfer interface can be adaptively displayed on the full screen (e.g., occupying the entire full screen) based on the display method established by the existing REE side. When the user enters the amount and clicks "next page" in the transfer interface, the TUI of the TA required by the mobile banking application is called out. Since the TUI cannot perceive the change of the screen, the TUI is still presented on the single screen (e.g., main screen) area in the full screen state. The other display areas of the full screen (e.g., the secondary screen area) are used to display other content, such as part of the wallpaper interface displayed by default.

[0197] contrast Figure 8 Neutral (1) and Figure 8 In (2), it is not difficult to see that when the display state of the screen changes, the UI interface on the REE side can change in the folding screen before and after the change, that is, it can be displayed adaptively. However, the TUI interface on the TEE side does not change in the folding screen before and after the change, and cannot be displayed adaptively based on the change of the folding screen, which brings a bad user experience.

[0198] The embodiments of the present invention can solve the above-mentioned problems and realize the adaptation of TUI to the folding screen. In this article, the so-called adaptation of TUI to the folding screen means that in response to the change of the display state of the folding screen, the interface of the TUI displayed on the folding screen also changes. The interface change of the TUI displayed on the folding screen may include a combination of one or more of the following changes: changes in the interface size of the TUI, changes in the position of the display area of ​​the TUI on the folding screen, changes in the interface shape of the TUI, changes in the interface color of the TUI, and changes (adjustments) in the content layout of the TUI on the folding screen.

[0199] It should be understood that in the embodiments of this document, the so-called TUI "adapted" to the folding screen and the so-called TUI "adaptively displayed" on the folding screen can be understood as the same meaning and are used in the document for the convenience of description.

[0200] For example, in one implementation, the TUI is adapted to the folding screen, and the size of the TUI interface changes in the folding screen before and after the change, and the TUI always occupies the screen corresponding to the display state of the folding screen. Figure 8 When the single screen is expanded to a full screen, the TUI interface can occupy the full screen, such as Fig. 9 The TUI shown in .

[0201] For example, in one implementation, the TUI is adapted to the folding screen by changing the position of the TUI display area in the folding screen before and after the change. For example, in the main screen display, the TUI interface occupies the entire main screen, such as Figure 8 When the main screen is expanded to full screen, the size of the TUI interface may not change, but the position of the TUI display area changes, such as Fig.10 In the TUI shown in FIG, the display area of ​​the TUI is moved to the middle position instead of the original main screen position. At this time, the TUI only occupies part of the screen in the full screen, rather than occupying the entire screen.

[0202] For example, in one implementation, the TUI is adapted to the folding screen by changing the shape of the TUI interface in the folding screen before and after the change. For example, in the main screen display, the TUI interface occupies the main screen, such as Figure 8 When the main screen is expanded to full screen, the interface shape of the TUI changes, as shown in (2) of FIG. Fig.11 The TUI shown in , in this case, the TUI can only occupy part of the screen in the full screen instead of occupying the entire screen.

[0203] For example, in one implementation, the TUI is adapted to the folding screen by changing the content layout of the TUI interface in the folding screen before and after the change. For example, in the main screen display, the TUI interface occupies the main screen, such as Figure 8 The TUI shown in (2) in FIG. 1 includes the bank name, receiving account number, amount, password input box, virtual keyboard, confirm / cancel key and other page contents. When the main screen is expanded to full screen, the interface shape of the TUI changes, and the interface contents also change, such as Fig.12 The TUI shown in the figure includes page contents such as a password input box, a virtual keyboard, and confirmation / cancel keys. At this time, the TUI can occupy only part of the screen in the full screen, rather than occupying the entire screen.

[0204] It should be noted that the above Figure 9-12 The embodiments are only used to explain the technical solution of the present invention and are not intended to be limiting. It should be understood that based on the technical concept of the present invention, other forms of TUIs adapted to foldable screens can also be designed in practical applications.

[0205] The following uses several specific implementation scenarios in mobile banking transfer scenarios as examples to explain the embodiments of the present invention.

[0206] like Fig.13 As shown, in a possible implementation scenario, a user logs in to a mobile banking application (i.e., CA) on the REE side in an electronic device, and the user interface (i.e., UI) of the mobile banking application is adaptively displayed on the current folding screen. Fig.13In the scenario given as an example, the folding screen of the electronic device is in a single-screen display state (for example, it can be a main screen display). The user enters the transfer amount and other information in the transfer interface of the UI. In order to facilitate the subsequent operations, the user unfolds the folding screen from a single screen (i.e., the first operation) to a full screen. The transfer interface can be adaptively displayed on the full screen based on the display method formulated by the existing REE side. In addition, in the processor of the electronic device, the REE side obtains the screen information of the full screen and transmits the screen information of the full screen to the TEE side. The user continues to click "Next Page" (i.e., the second operation) in the transfer interface, so that the TUI of the TA required by the mobile banking application is called out, and the TEE side processes the TUI according to the screen information of the full screen (such as coordinate conversion, TUI content drawing, etc.), and displays the TUI in the full screen through a specific interface call, as shown in the figure, the TUI includes an interface for PIN code input and verification. At this time, the interface of the TUI displayed on the full screen is adapted to the full screen, as shown in the figure, the interface of the TUI occupies the entire full screen. The user can perform further operations on the TUI, such as inputting a PIN code and confirming transaction information. After the user completes the input of the PIN code and confirmation of the transaction information through the TUI, the full-screen display area is returned to REE, and the mobile banking application on the REE side then displays the transfer completion interface, and the entire transaction process is completed.

[0207] like Fig.14 As shown, in another possible implementation scenario, a user logs in to a mobile banking application (ie, CA) on the REE side in an electronic device, and the user interface (ie, UI) of the mobile banking application is adaptively displayed on the current folding screen. Fig.14In the scenario given as an example, the folding screen of the electronic device is in a single-screen display state (for example, it can be a main screen display). The user enters the transfer amount and other information in the transfer interface of the UI. The user continues to click "Next Page" in the transfer interface, so that the TUI of the TA required by the mobile banking application is called out, and the TEE side calls the TUI through a specific interface to display it in the single screen. As shown in the figure, the TUI includes an interface for PIN code input and verification. In order to facilitate input, the user expands the folding screen from a single screen (i.e., the first operation) to a full screen. In the processor of the electronic device, the REE side obtains the screen information of the full screen and transmits the screen information of the full screen to the TEE side. In order to ensure information security, the TUI automatically exits, and the full screen will display the interface of the most recent UI before the TUI, such as the transfer interface shown in the figure, and the UI is adaptively presented in the full screen. The user continues to click "Next Page" (i.e., the second operation) in the transfer interface, so that the TUI of the TA required by the mobile banking application is called out. The TEE side processes the TUI according to the screen information of the full screen, and displays the TUI in the full screen through a specific interface call. As shown in the figure, the TUI includes an interface for PIN code input and verification. At this time, the interface of the TUI displayed on the full screen is adapted to the full screen, as shown in the figure, the interface of the TUI occupies the entire full screen. The user can perform further operations on the TUI, such as inputting the PIN code and confirming the transaction information. After the user completes the input of the PIN code and confirmation of the transaction information through the TUI, the display area of ​​the full screen is returned to the REE, and the mobile banking application on the REE side then displays the transfer completion interface, and the entire transaction process ends.

[0208] like Fig.15 As shown, in another possible implementation scenario, a user logs in to a mobile banking application (ie, CA) on the REE side in an electronic device, and the user interface (ie, UI) of the mobile banking application is adaptively displayed on the current folding screen. Fig.15In the scenario given as an example, the folding screen of the electronic device is in a single-screen display state (for example, it can be a main screen display). The user enters the transfer amount and other information in the transfer interface of the UI. The user continues to click "Next Page" in the transfer interface, so that the TUI of the TA required by the mobile banking application is called out, and the TEE side calls the TUI through a specific interface to display it in the single screen, as shown in the figure, the TUI includes an interface for PIN code input and verification. In order to facilitate input, the user unfolds the folding screen from a single screen to a full screen. In the processor of the electronic device, the REE side obtains the screen information of the full screen and transmits the screen information of the full screen to the TEE side. The TEE side processes the TUI according to the screen information of the full screen, such as redrawing the TUI, and calling the specific interface to display the newly drawn TUI in the full screen, as shown in the figure, the TUI includes an interface for PIN code input and verification. At this time, the interface of the TUI displayed on the full screen is adapted to the full screen, as shown in the figure, the interface of the TUI occupies the entire full screen. The user can perform further operations on the TUI, such as inputting a PIN code and confirming transaction information. After the user completes the input of the PIN code and confirmation of the transaction information through the TUI, the full-screen display area is returned to REE, and the mobile banking application on the REE side then displays the transfer completion interface, and the entire transaction process is completed.

[0209] It should be noted that the above Figure 13-15 The embodiments are only used to exemplify the technical solution of the present invention and are not intended to limit the present invention. Based on the technical concept of the present invention, more implementation processes (such as other transfer processes) can be derived in practical applications.

[0210] Based on the above description, some TUI display methods provided by embodiments of the present invention are further described below.

[0211] For the sake of convenience, the various method embodiments described below are expressed as a combination of a series of action steps, but those skilled in the art should be aware that the specific implementation of the technical solution of the present application is not limited by the order of the described series of action steps.

[0212] See also Fig.16 , Fig.16 is a flow chart of a TUI display method provided by an embodiment of the present invention, which is described from the perspectives of the REE side and the TEE side. The method can be applied to Figure 5 The method includes but is not limited to the following steps:

[0213] 1. The screen status perception module on the REE side obtains the screen information of the current folding screen by monitoring the system broadcast.

[0214] Specifically, when the display state of the folding screen changes, the screen state perception module can obtain the screen information of the current folding screen by monitoring the system broadcast (for example, Android system broadcast).

[0215] For example, a user operates the folding screen so that the display state of the screen changes from full display to single-screen display (that is, the folding screen changes from an unfolded state to a folded state), or from single-screen display to full-screen display (that is, the folding screen changes from a folded state to an unfolded state), or from one single-screen display to another single-screen display (for example, flipping the electronic device so that the folding screen changes from a main screen display to a secondary screen display state).

[0216] The screen information of the current folding screen may include, for example, the resolution of the current folding screen (ie, the resolution of the screen corresponding to the display state), and further, may include the switch state of each screen of the folding screen (such as the start / stop state).

[0217] 2. The screen status perception module transmits the screen information to the adaptive display module on the TEE side.

[0218] The screen status sensing module may be a system service process (daemon).

[0219] Specifically, the screen status perception module can transmit the screen information to the adaptive display module on the TEE side through the driver module in the REE (such as the TUI proxy function or the TUI conversion function).

[0220] 3. The adaptive display module updates the local screen information.

[0221] Specifically, the adaptive display module dynamically saves the screen information sent from the REE side, that is, when receiving the screen information, the new screen information is saved locally to replace the screen information received last time.

[0222] It should be noted that step 3 is optional.

[0223] 4. The adaptive display module obtains coordinate conversion information according to the screen information.

[0224] The coordinate conversion information indicates the coordinate transformation relationship between the drawing coordinate system and the physical coordinate system.

[0225] Specifically, the adaptive display module can determine the physical coordinate system of the current folding screen according to the screen information, and obtain the coordinate conversion information between the two coordinate systems according to the saved or default drawing coordinate system and the physical coordinate system.

[0226] Among them, the physical coordinate system can also be called the device coordinate system, which refers to the actual coordinates of the entity based on the electronic device. For example, in the embodiment of the present invention, it can specifically refer to the physical coordinates on the screen of the current folding screen on the electronic device. The physical coordinate system used for the screen defines the physical origin, X-axis, and Y-axis to determine the object on the screen. In such a coordinate system, the coordinates of an object (such as a pixel) include the distance of the object from the X-axis and the distance from the Y-axis, that is, the position of the object can be determined according to the coordinates of the object, where the coordinates can be expressed in pixels. For example, the physical coordinate system can be used for an electronic scanner to display interface content on the screen.

[0227] In the embodiment of the present invention, the physical coordinate systems used for different display states of the folding screen are also different. Fig.17A , Fig.17A Figure 1 is a simple example of the physical coordinate system used in different display states of the folding screen. The folding screen includes a secondary screen, a folding edge, and a main screen. Fig.17A In the scene shown, physical origin 1 represents the physical origin when the secondary screen is displayed, physical origin 2 represents the physical origin when the main screen is displayed, and physical origin 2 can also represent the physical origin when the full screen is displayed. That is to say, in the folded state, when the secondary screen is displayed, the physical origin of the physical coordinate system is located at the upper right corner of the secondary screen, and when the main screen is displayed, the physical origin of the physical coordinate system is located at the upper right corner of the main screen. In the expanded state, when the full screen is displayed, the physical origin of the physical coordinate system is located at the upper right corner of the full screen. In addition, the X-axis of the physical coordinate system is positive downward, and the Y-axis is positive to the left.

[0228] The drawing coordinate system is also called the logical coordinate system. It is the coordinate system used by the system for recording or drawing. It defines the logical origin, X-axis, and Y-axis used for recording or drawing. For example, see Fig. 17B , Fig. 17B This is a simple example of the drawing coordinate system used in different display states of the folding screen. Fig. 17B In the scene shown, logical origin 1 represents the logical origin for the secondary screen, and logical origin 1 can also represent the logical origin for the full screen. Origin 2 represents the logical origin for the main screen. The X axis of the drawing coordinate system is positive to the right, and the Y axis is positive downward. The coordinates are also expressed in pixels.

[0229] Specifically, the adaptive display module can determine the physical coordinate system used by the current folding screen according to the latest received screen information, and then determine the coordinate conversion relationship between the drawing coordinate system and the physical coordinate system.

[0230] For example, the screen information includes the resolution information of the current folding screen, such as "Height = 2200, Width = 2440", and then according to the resolution information, it is determined that the current folding screen is in a full-screen display state. Then, the adaptive display module can determine the coordinate conversion information between the drawing coordinate system and the physical coordinate system used for full-screen display.

[0231] For example, in the scenario of screen display, the coordinate conversion information includes "x=y3, y= Width-x3", where {x, y} represents physical coordinates and {x3, y3} represents drawing coordinates, so as to adapt the TUI interface to the current folding screen.

[0232] For another example, in the scenario of user command input, the coordinate conversion information includes "x3= Width-y, y3=x", where {x, y} represents physical coordinates and {x3, y3} represents drawing coordinates, so that the user's command input can be matched to the TUI control event through coordinate conversion.

[0233] 5. When TA's TUI is called, the adaptive display module redraws the TUI according to the coordinate conversion information, so that the TUI is adapted to the current folding screen.

[0234] Specifically, when the TA's TUI is called, the adaptive display module processes the interface of the original default TUI (or preset TUI) according to the coordinate conversion information, so that the coordinates of each pixel point in the TUI interface (i.e., the coordinates in the drawing coordinate system) are converted into coordinates in the physical coordinate system, thereby redrawing the TUI and obtaining the redrawn TUI. Since the coordinates of each pixel point in the redrawn TUI are all converted into coordinates in the physical coordinate system, when the redrawn TUI is displayed on the current folding screen, it can occupy the entire screen, thereby achieving the adaptation of the TUI to the current folding screen.

[0235] Furthermore, when the redrawn TUI needs to be scaled, the adaptive display module can also scale the TUI interface according to the coordinate conversion information.

[0236] For example, in the scenario of screen display, the coordinate conversion information includes "x=y3,y= Width-x3", where {x, y} represents physical coordinates, and {x3, y3} represents drawing coordinates. When scaling is required, the coordinate conversion information is adjusted to "x=y3*k1,y= (Width-x3)*k2", k1 represents the scale factor, and k2 represents the scale factor, for example, k1=50%, k2=50%; k1=60%, k2=80%; k1=40%, k2=00%; k1=60%, k2=40%, and so on, without limitation here.

[0237] For example, in the scenario of user command input, the coordinate conversion information includes "x3= Width-y, y3=x", where {x, y} represents physical coordinates and {x3, y3} represents drawing coordinates. When scaling is required, the coordinate conversion information is adjusted to "x3= (Width-y) * k1, y3=x*k2", where k1 represents the scale factor and k2 represents the scale factor, for example, k1=50%, k2=50%; k1=60%, k2=80%; k1=40%, k2=00%; k1=60%, k2=40%, etc., without limitation here.

[0238] After scaling, the coordinates of each pixel in the TUI are converted into the coordinates of the physical coordinate system, and the interface size of the TUI has changed compared to the original default TUI. When the TUI is displayed on the current folding screen, it can occupy part of the screen, thereby making the TUI adapt to the current folding screen.

[0239] In addition, the adaptive display module can also perform further processing on the TUI interface based on the coordinate conversion information, so that compared with the original default TUI, the redrawn TUI not only has the coordinates of each pixel point converted into the coordinates of the physical coordinate system, but also the page undergoes one or more of the following changes: changes in the interface size of the TUI, changes in the position of the display area of ​​the TUI in the folding screen, changes in the interface shape of the TUI, changes in the interface color of the TUI, changes (adjustments) in the content layout of the TUI in the folding screen, thereby realizing various adaptation schemes for the TUI to adapt to the current folding screen.

[0240] Optionally, the redrawn TUI may also support different display orientations, such as landscape orientation and portrait orientation.

[0241] It can be seen that in the embodiment of the present invention, for an electronic device with a folding screen, when the display state of the folding screen where the application service is located changes, the screen information of the current folding screen (the latest screen information) can be transmitted to the adaptive display module on the TEE side through the screen state perception module, so that the TEE side can obtain the screen information of the folding screen in real time, creating a feasible information transmission channel; the screen information is processed by the adaptive display module to realize the automatic adaptation of the TUI interface to the screen of the current folding screen, and the adaptive display of TUI is realized by using software algorithms (rather than GPU display), which can solve the pain points of folding screen users and greatly improve the user experience.

[0242] See also Fig.18 , Fig.18is a flow chart of another TUI display method provided by an embodiment of the present invention, which is described from the perspectives of the REE side and the TEE side. The method can be applied to Figure 6 The method includes but is not limited to the following steps:

[0243] 1. The screen status perception module on the REE side obtains the screen information of the current folding screen by monitoring the system broadcast.

[0244] For reference Fig.16 The description of step 1 in is not repeated here.

[0245] 2. The screen status perception module sends the encrypted screen information to the identity authentication module on the REE side.

[0246] For example, the screen status perception module encrypts the screen information of the current folding screen through the session-key to obtain the encrypted screen information.

[0247] Furthermore, the screen information may also include a challenge value (or a random number), and the screen state sensing module encrypts the screen information including the challenge value (or the random number) to obtain encrypted screen information.

[0248] Alternatively, optionally, the screen state sensing module encrypts the screen information. Then, the screen state sensing module may encapsulate the encrypted screen information and the challenge value (or random number) in a data packet.

[0249] It should be noted that the screen information may also be encrypted in other ways (eg, symmetric encryption), which is not limited here.

[0250] 3. The identity authentication module performs identity authentication based on the screen information.

[0251] The identity authentication module verifies the identity of the information transmitter (here, the screen status perception module) to ensure that only system services can call the information transmission interface to send screen information to the TEE side.

[0252] Specifically, the identity authentication module can obtain the identity information of the information transmitter through the process information (process) to determine whether the information transmitter is a system service. In this way, when the information transmitter is the screen status perception module designed in the embodiment of this article, since the identity authentication module can determine that the screen status perception module is a system service, the identity authentication module will allow the screen status perception module to call the information transmission interface (call the TUI-switch function or the TUI-agent function). When the information transmitter is a function code forged by a malicious attacker, the identity authentication module confirms that the information transmitter is not a system service and will refuse the information transmitter to call the relevant interface to prevent unsafe information from being transmitted to the TEE side, thereby ensuring the information security of the REE side and the TEE side.

[0253] 4. The authentication module transmits the screen information to the session key verification module on the TEE side.

[0254] Specifically, the identity authentication module can transmit the encrypted screen information (or data packet) to the session key verification module on the TEE side through the TUI agent.

[0255] 5. The session key verification module performs identity authentication based on the screen information (or data packet).

[0256] Specifically, the session key verification module decrypts the screen information and verifies the identity information of the information transmitter based on the challenge value (random number) carried by it. For example, when the challenge value (random number) is verified correctly, it is confirmed that the identity of the information transmitter is legal, that is, it is confirmed that the information transmitter is the screen status perception module designed in the embodiment of the present application, rather than a function code disguised by a malicious attacker. This verification can further improve the information security on the TEE side, for example, to ensure that the screen information will not be tampered with after coming out of the aforementioned identity authentication module.

[0257] 6. The session key verification module transmits the decrypted screen information to the adaptive display module on the TEE side.

[0258] Specifically, when the challenge value (random number) verification and the key verification are both correct, the session key verification module transmits the decrypted screen information to the adaptive display module on the TEE side.

[0259] 7. The adaptive display module updates the local screen information.

[0260] Specifically, the adaptive display module dynamically saves the screen information sent from the REE side, that is, when receiving the screen information, the new screen information is saved locally to replace the screen information received last time.

[0261] It should be noted that step 7 is optional.

[0262] 8. The adaptive display module performs coordinate conversion processing according to the decrypted screen information to obtain processed screen information.

[0263] For reference Fig.16 The description of step 4 in is not repeated here.

[0264] 9. When TA's TUI is called, the adaptive display module redraws the TUI according to the coordinate conversion information, so that the TUI is adapted to the current folding screen.

[0265] For reference Fig.16 The description of step 5 in is not repeated here.

[0266] It should be noted that when the functions of the session key verification module and the adaptive display module are deployed together, the above step 6 is optional.

[0267] The following further Fig.19 Take the following example to explain the solution. Fig.19 As shown, the user operates the electronic device, causing the display state of the folding screen to change. After the screen state perception module monitors it, it passes the encrypted (for example, session-key encrypted) screen information and random numbers to the authentication module of REE for authentication, ensuring that only system services can continue to send down. After the verification is passed, the encrypted screen information is further processed by the hardware abstraction layer interface definition language (HALinterface definition language, HIDL), and then further transmitted to the TUI agent module. The TUI agent module sends the encrypted screen information to the TEE side. On the TEE side, after the global task module obtains the encrypted screen information, it further sends the encrypted screen information to the session key verification module for identity authentication, such as verifying the random number and verifying the session-key. After the verification is passed, the adaptive display module obtains the screen information and realizes the adaptive display of TA's TUI according to the screen information.

[0268] It can be seen that in the embodiment of the present invention, for an electronic device with a folding screen, when the display state of the folding screen where the application service is located changes, the screen information of the current folding screen (the latest screen information) can be encrypted and transmitted to the adaptive display module on the TEE side through the screen state perception module, so that the TEE side can obtain the screen information of the folding screen in real time. And in this process, the identity authentication module is used to perform preliminary identity authentication on the REE side, and the session key verification module is used to perform further identity authentication on the TEE side, creating a safe and reliable information transmission channel to ensure the information security of the REE and TEE sides. The screen information is processed by the adaptive display module to realize the automatic adaptation of the TUI interface to the current folding screen, and the adaptive display of the TUI is realized by the software algorithm (rather than the GPU display method), which can solve the pain points of folding screen users and greatly improve the user experience.

[0269] See also Fig. 20 , Fig. 20 This is a flow chart of a TUI display method provided by an embodiment of the present invention, which is described from the perspective of the electronic device. The method includes but is not limited to the following steps:

[0270] 1. In response to a first operation of a user on an electronic device with a folding screen, the folding screen is changed from a first display state to a second display state.

[0271] The first display state indicates that at least one of the at least two screens is in the display state, the second display state indicates that at least one of the at least two screens is in the display state, and the screen corresponding to the first display state is different from the screen corresponding to the second display state.

[0272] The relevant contents regarding the electronic device, the folding screen, the first operation, the first display state of the folding screen, the second display state of the folding screen, and the change from the first display state to the second display state have been fully described in the previous text and related drawings. For the sake of brevity of the specification, they will not be repeated here.

[0273] 2. Display the user interface UI of CA on the screen corresponding to the second display state.

[0274] The relevant contents about the screen, CA and UI corresponding to the second display state have been fully described in the previous text and related drawings. The specific implementation details / processes / examples of this step have also been fully described in the previous text and related drawings. For the sake of brevity of the specification, they will not be repeated here.

[0275] 3. In response to a second operation of the user on the UI of the CA, a trusted user interface TUI corresponding to the CA is triggered on the screen corresponding to the second display state, and the TUI is adapted to the screen corresponding to the second display state.

[0276] The second operation, the TUI corresponding to CA, and the adaptation of the TUI to the screen corresponding to the second display state (i.e., the current folded screen) have been fully described in the previous text and related drawings. The specific implementation details / processes / examples of this step have also been fully described in the previous text and related drawings. For the sake of brevity, they will not be repeated here.

[0277] It can be seen that in the embodiment of the present invention, for an electronic device with a foldable screen, when the display state of the foldable screen where the application service is located changes, the embodiment of the present invention can realize both the adaptive display of CA's UI on the foldable screen and the adaptive display of TA's TUI on the foldable screen, which can solve the pain points of foldable screen users and greatly improve the user experience.

[0278] See also Fig.21 , Fig.21 This is a flow chart of a TUI display method provided by an embodiment of the present invention, which is described from the perspective of the electronic device. The method includes but is not limited to the following steps:

[0279] 1. Display the user interface UI of CA on the screen corresponding to the first display state of the folding screen, wherein the first display state indicates that at least one of the at least two screens is in the display state.

[0280] The relevant contents about the first display state, CA, and UI of the folding screen have been fully described in the previous text and related drawings. The specific implementation details / processes / examples of this step have also been fully described in the previous text and related drawings. For the sake of brevity of the specification, they will not be repeated here.

[0281] 2. In response to a second operation performed by the user on the CA's UI, triggering display of the trusted user interface TUI corresponding to the CA on the screen corresponding to the first display state.

[0282] The relevant contents about the second operation, the second display state, and the TUI corresponding to CA have been fully described in the previous text and related drawings. The specific implementation details / processes / examples of this step have also been fully described in the previous text and related drawings. For the sake of brevity of the specification, they will not be repeated here.

[0283] 3. In response to a first operation of the user on the electronic device, determine to change the folding screen from a first display state to a second display state. The second display state indicates that at least one of the at least two screens is in a display state, and the screen corresponding to the first display state is different from the screen corresponding to the second display state.

[0284] The relevant contents regarding the first operation on the electronic device and the change of the first display state to the second display state have been fully described in the previous text and related drawings. The specific implementation details / processes / examples of this step have also been fully described in the previous text and related drawings. For the sake of brevity of the specification, they will not be repeated here.

[0285] 4. In response to the user's first operation on the CA's UI, trigger the display of the trusted user interface TUI corresponding to the CA on the screen corresponding to the second display state, and the TUI is adapted to the screen corresponding to the second display state.

[0286] The first operation, the TUI corresponding to CA, and the adaptation of the TUI to the screen corresponding to the second display state (i.e., the current folded screen) have been fully described in the previous text and related drawings. The specific implementation details / processes / examples of this step have also been fully described in the previous text and related drawings. For the sake of brevity, they will not be repeated here.

[0287] It can be seen that in the embodiment of the present invention, for an electronic device with a foldable screen, when the display state of the foldable screen where the application service is located changes, the embodiment of the present invention can realize both the adaptive display of CA's UI on the foldable screen and the adaptive display of TA's TUI on the foldable screen, which can solve the pain points of foldable screen users and greatly improve the user experience.

[0288] The above describes the relevant system architecture and method of the embodiment of the present invention. Based on the same inventive concept, the following describes the relevant device / equipment of the embodiment of the present invention.

[0289] Fig. 22 An example of an electronic device 300 provided by an embodiment of the present invention is shown. Fig. 22 , the electronic device 300 includes a communication subsystem 210, a power supply 220, an input device 230, a display device 240, a processing unit 250, and a memory 260. The memory 260 stores a computer program or instruction, and the computer program includes an operating system 294 and an application 292, etc. The processing unit 250 is configured to execute the computer program in the memory 260, so as to implement the method defined by the computer program, for example, the processing unit 250 runs the operating system 294 to implement various functions of the REE module and the TEE module described in the above embodiments on the electronic device 300.

[0290] The processing unit 250 may include one or more processors. For example, the processing unit 250 may include an application processor, a graphics processor, a digital signal processor, etc. When the processing unit 250 includes multiple processors, the multiple processors may be integrated on the same chip or may be independent chips.

[0291] The memory 260 also stores other data 296 besides computer programs. The other data 296 may include data generated during the operation of the operating system 294 or the application program 292, such as system data (eg, configuration parameters of the operating system 294) and user data.

[0292] The memory 260 generally includes internal memory and external memory. The internal memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), or cache, etc. The external memory includes, but is not limited to, flash memory, hard disk, universal serial bus (USB) disk, etc. Computer programs are usually stored in the external memory, and the processing unit 250 will load the program from the external memory to the internal memory before executing the computer program.

[0293] In one embodiment, the operating system 294 includes a computer program for implementing the TUI display method provided in the embodiment of the present invention, so that after the processor 250 runs the operating system 294, the steps of the TUI display method provided in the embodiment of the present invention are implemented. Fig.16 , Fig.18 , Fig. 20 , Fig.21 Relevant method steps in the electronic device shown in the embodiment. Exemplarily, the view manager 173, layer synthesizer 175, TUI service 150, and layer synthesizer 160 described in the above embodiment can be implemented in the form of computer programs (instructions), and the processing unit 250 loads and runs these computer programs (instructions) to implement the functions of these modules.

[0294] The input device 230 is used to receive user input information, such as digital / character information, touch operations or gestures, and generate corresponding input signals. Specifically, in one embodiment, the input device 230 includes a touch panel. A touch panel, also known as a touch screen, can collect user touch operations thereon and generate touch signals to drive related components to respond to user operations. In addition to the touch panel, the input device 230 may also include other input devices, including but not limited to one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

[0295] The display device 240 may be a display panel such as a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc. In some embodiments, a touch panel may cover the display device 240 to form a touch display screen. The display device 240 is used to display images, videos, etc. The display device 240 includes a folding screen (such as a flexible folding screen or a multi-screen folding screen) described in each embodiment of this document. The display device 240 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light emitting diode (QLED), etc.

[0296] The communication subsystem 210 is a basic communication unit of the terminal 200, and is used for data transmission and reception of the terminal 200. The power supply 220 is used to supply power to the above-mentioned components, and may specifically be a power management chip.

[0297] When the electronic device 300 is a wireless terminal, the communication subsystem 210 includes a wireless modem, which mainly implements functions such as baseband processing, modulation and demodulation, signal amplification and filtering, and equalization. In one embodiment, the communication subsystem 210 includes: a baseband processor, a radio frequency circuit, and an antenna. Among them, the radio frequency circuit and the antenna are mainly responsible for signal transmission and reception; the baseband processor is responsible for signal processing, such as signal A / D, D / A conversion, signal encoding and decoding, etc. The baseband processor supports one or more wireless communication standards, where the wireless communication standards include but are not limited to the global system for mobile communications (GSM), code division multiple access (CDMA), wideband code division multiple access (WCDMA), high speed packet access (HSPA), long-term evolution (LTE), etc. The baseband processor can be a separate chip, or it can be integrated in the same chip with the processor included in the processing unit 250.

[0298] Optionally, the electronic device 300 further includes one or more sensors 280 , such as an acceleration sensor, a light sensor, etc.

[0299] The TUI display method provided by the embodiment of the present invention may be executed by a proper combination of software, hardware and / or firmware of the electronic device 300 .

[0300] In addition, those skilled in the art will appreciate that the terminal 200 may include Fig. 22 Fewer or more parts than shown in Fig. 22 The electronic device 300 shown only shows components that are more relevant to multiple implementations disclosed in the embodiments of the present invention.

[0301] See also Fig.23 , Fig.23 80 is a schematic diagram of a structure of a device 80 provided in an embodiment of the present invention, and the device 80 includes a REE module 801 and a TEE module 802. In some embodiments, the REE module 801 and the TEE module 802 can be run on the processing unit 250. Among them:

[0302] The REE module 801 is used to, in response to a first operation of the user on the electronic device, determine that the folding screen changes from a first display state to a second display state; wherein the first display state indicates that at least one of the at least two screens is in a display state, and the second display state indicates that at least one of the at least two screens is in a display state, and the screen corresponding to the first display state is different from the screen corresponding to the second display state;

[0303] The REE module 801 is further configured to display the user interface UI of the CA on the screen corresponding to the second display state;

[0304] The TEE module 802 is used for, in response to the second operation of the user on the UI of the CA, triggering display of the trusted user interface TUI corresponding to the CA on the screen corresponding to the second display state, and the TUI is adapted to the screen corresponding to the second display state.

[0305] REE module 801 and TEE module 802 can be used to implement Fig.16 , Fig.18 , Fig. 20 , Fig.21 For the sake of brevity of the specification, the relevant method steps in the electronic device shown in the embodiment are not repeated here.

[0306] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0307] In the several embodiments provided by the present invention, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of devices or units can be electrical or other forms.

[0308] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0309] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0310] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a memory, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0311] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable memory, which may include a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

Claims

1. A method for displaying a trusted user interface, characterized in that: The method is applied to an electronic device with a folding screen, the folding screen includes at least two screens, a client application CA runs on the electronic device, a rich execution environment REE module and a trusted execution environment TEE module are deployed on the electronic device, and the CA runs on the REE module; The trusted application TA corresponding to the CA runs on the TEE module, and the method includes: In response to a first operation of the user on the electronic device, the folding screen is changed from a first display state to a second display state; wherein the first display state indicates that at least one of the at least two screens is in a display state, and the second display state indicates that at least one of the at least two screens is in a display state, and the screen corresponding to the first display state is different from the screen corresponding to the second display state; Displaying the user interface UI of the CA on the screen corresponding to the second display state through the REE module; Through the TEE module, in response to the second operation of the user on the UI of the CA, a trusted user interface TUI corresponding to the CA is triggered to be displayed on the screen corresponding to the second display state, and the TUI is adapted to the screen corresponding to the second display state.

2. The method according to claim 1, characterized in that: The first operation is an operation of unfolding the folding screen to display the full screen; the first display state indicates that one of the at least two screens is in a display state, and the second display state indicates that the at least two screens are in a display state.

3. The method according to claim 1, characterized in that The first operation is an operation of folding the folding screen into a single-screen display; the first display state indicates that the at least two screens are in a display state, and the second display state indicates that one of the at least two screens is in a display state.

4. The method according to claim 1, characterized in that: The first operation is an operation of flipping the electronic device to switch the folding screen from a first single-screen display to a second single-screen display; the first display state indicates that the first screen among the at least two screens is in a display state, and the second display state indicates that the second screen among the at least two screens is in a display state.

5. The method according to any one of claims 1 to 4, characterized in that: After triggering display of the TUI of the CA on the screen corresponding to the second display state, the method further includes: In response to a third operation of the user on the electronic device, determining that the folding screen is changed from the second display state to a third display state, wherein the third display state indicates that at least one of the at least two screens is in a display state, and a screen corresponding to the third display state is different from a screen corresponding to the second display state; exiting the TUI in the screen corresponding to the third display state, and displaying the UI of the CA; In response to a fourth operation of the user on the UI of the CA, the TUI of the CA is triggered to be displayed on the screen corresponding to the third display state, and the TUI is adapted to the screen corresponding to the third display state.

6. The method according to claim 5, characterized in that The fourth operation is the same as the second operation.

7. The method according to claim 5, characterized in that The screen corresponding to the third display state is the same as the screen corresponding to the first display state; or The screen corresponding to the third display state is different from the screen corresponding to the first display state.

8. The method according to any one of claims 1 to 4 or 6 or 7, characterized in that: The adaptation of the TUI to the screen corresponding to the second display state means that the TUI occupies the entire screen on the screen corresponding to the second display state.

9. The method according to any one of claims 1 to 4 or 6 or 7, characterized in that: The adaptation of the TUI to the screen corresponding to the second display state means that the TUI occupies a part of the screen on the screen corresponding to the second display state.

10. The method according to claim 1, characterized in that The step of determining, in response to a first operation of the user on the electronic device, that the folding screen changes from a first display state to a second display state includes: When the folding screen changes from the first display state to the second display state, obtaining, through the REE module, screen information corresponding to the folding screen in the second display state; The screen information is transmitted to the TEE module through the REE module.

11. The method according to claim 10, characterized in that The triggering, in response to the second operation of the user on the UI of the CA, to display the TUI of the TA in the TEE module on the screen corresponding to the second display state, wherein the TUI is adapted to the screen corresponding to the second display state, comprises: triggering, by the REE module, a TUI display request in response to a second operation of the user on the UI of the CA; The TUI display request is sent to the TEE module through the REE module, so as to call the TA corresponding to the CA in the TEE module.

12. The method according to claim 11, characterized in that The triggering, in response to the second operation of the user on the UI of the CA, to display the TUI of the TA in the TEE module on the screen corresponding to the second display state, wherein the TUI is adapted to the screen corresponding to the second display state, comprises: Receiving the TUI display request through the TEE module; Drawing the TUI of the TA in the TEE module according to the screen information through the TEE module, so that the TUI is adapted to the screen corresponding to the second display state; The TUI is displayed on the screen corresponding to the second display state through the TEE module.

13. The method according to claim 12, characterized in that The step of drawing the TUI of the TA in the TEE module according to the screen information so that the TUI is adapted to the screen corresponding to the second display state includes: Obtaining coordinate transformation information according to the screen information through the TEE module; The TUI is drawn according to the coordinate transformation information through the TEE module, and the size of the TUI is consistent with the size of the screen corresponding to the second display state, or the size of the TUI is smaller than the size of the screen corresponding to the second display state.

14. The method according to any one of claims 10, 12 or 13, characterized in that: The transmitting the screen information to the TEE module through the REE module includes: Encrypting the screen information through the REE module to obtain encrypted screen information; Verifying, by the REE module, whether the process for transmitting the encrypted screen information is a system service; Through the REE module, when the process is a system service, a driver interface is called to transmit the encrypted screen information to the TEE module.

15. The method according to any one of claims 10, 12 or 13, characterized in that: The transmitting the screen information to the TEE module through the REE module includes: Verifying, by the TEE module, the identity of the process used to transmit the screen information according to the screen information; Through the TEE module, when the verification is passed, the screen information is decrypted and obtained.

16. An electronic device with a foldable screen, characterized in that: The folding screen includes at least two screens, a client application CA runs on the electronic device, a rich execution environment REE module and a trusted execution environment TEE module are deployed on the electronic device, and the CA runs on the REE module; The trusted application TA corresponding to the CA runs on the TEE module, including: A rich execution environment (REE) module, configured to change the folding screen from a first display state to a second display state in response to a first operation of the user on the electronic device; wherein the first display state indicates that at least one of the at least two screens is in a display state, and the second display state indicates that at least one of the at least two screens is in a display state, and the screen corresponding to the first display state is different from the screen corresponding to the second display state; and further configured to display a user interface (UI) of the CA on the screen corresponding to the second display state; A trusted execution environment TEE module is used to trigger the display of a trusted user interface TUI corresponding to the CA on the screen corresponding to the second display state in response to the second operation of the user on the UI of the CA, and the TUI is adapted to the screen corresponding to the second display state.

17. The electronic device according to claim 16, characterized in that: The first operation is an operation of unfolding the folding screen to display the full screen; the first display state indicates that one of the at least two screens is in a display state, and the second display state indicates that the at least two screens are in a display state.

18. The electronic device according to claim 16, characterized in that: The first operation is an operation of folding the folding screen into a single-screen display; the first display state indicates that the at least two screens are in a display state, and the second display state indicates that one of the at least two screens is in a display state.

19. The electronic device according to claim 16, characterized in that: The first operation is an operation of flipping the electronic device to switch the folding screen from a first single-screen display to a second single-screen display; the first display state indicates that the first screen among the at least two screens is in a display state, and the second display state indicates that the second screen among the at least two screens is in a display state.

20. The electronic device according to any one of claims 16 to 19, wherein the REE module is further used for: In response to a third operation of the user on the electronic device, determining that the folding screen is changed from the second display state to the third display state, the third display state indicating that at least one of the at least two screens is in a display state, and the screen corresponding to the third display state is different from the screen corresponding to the second display state; after the TUI in the screen corresponding to the third display state is exited, displaying the UI of the CA; The TEE module is also used to: In response to a fourth operation of the user on the UI of the CA, the TUI of the CA is triggered to be displayed on the screen corresponding to the third display state, and the TUI is adapted to the screen corresponding to the third display state.

21. The electronic device according to claim 20, characterized in that: The fourth operation is the same as the second operation.

22. The electronic device according to claim 20, characterized in that: The screen corresponding to the third display state is the same as the screen corresponding to the first display state; or The screen corresponding to the third display state is different from the screen corresponding to the first display state.

23. The electronic device according to any one of claims 16 to 19 or 21 or 22, characterized in that: The adaptation of the TUI to the screen corresponding to the second display state means that the TUI occupies the entire screen on the screen corresponding to the second display state.

24. The electronic device according to any one of claims 16 to 19 or 21 or 22, characterized in that: The adaptation of the TUI to the screen corresponding to the second display state means that the TUI occupies a part of the screen on the screen corresponding to the second display state.

25. The electronic device according to claim 16, characterized in that: The REE module is specifically used for: When the folding screen changes from the first display state to the second display state, obtaining screen information corresponding to the folding screen in the second display state; The screen information is transmitted to the TEE module.

26. The electronic device according to claim 25, characterized in that: The REE module is specifically used for: In response to a second operation of the user on the UI of the CA, triggering a TUI display request; The TUI display request is sent to the TEE module, so as to call the TA corresponding to the CA in the TEE module.

27. The electronic device according to claim 26, characterized in that: The TEE module is specifically used for: receiving the TUI display request; Draw the TUI of the TA in the TEE module according to the screen information, so that the TUI is adapted to the screen corresponding to the second display state; The TUI is displayed on a screen corresponding to the second display state.

28. The electronic device according to claim 27, characterized in that: The TEE module is specifically used for: Obtaining coordinate transformation information according to the screen information; The TUI is drawn according to the coordinate transformation information, and the size of the TUI is consistent with the size of the screen corresponding to the second display state, or the size of the TUI is smaller than the size of the screen corresponding to the second display state.

29. The electronic device according to any one of claims 25, 27 or 28, characterized in that: The REE module is specifically used for: Encrypting the screen information to obtain encrypted screen information; Verifying whether the process for transmitting the encrypted screen information is a system service; In the case where the process is a system service, a driver interface is called to transmit the encrypted screen information to the TEE module.

30. The electronic device according to any one of claims 25, 27 or 28, characterized in that: The REE module is specifically used for: verifying the identity of the process used to transmit the screen information based on the screen information; When the verification is passed, the screen information is decrypted and obtained.

Images