User interface manipulation in foldable screen device

Abstract

To provide user interface (UI) manipulation technology that enables a user device to hide from bystanders, or make obscure, confidential information displayed on a display which is reconfigurable by flexible, foldable, or other methods on one hand, and enables maintaining or improving exclusive accessibility to main users on the other hand.SOLUTION: User interface (UI) manipulation techniques are disclosed that can allow a user device to hide and obscure sensitive information displayed on a flexible, foldable, or otherwise reconfigurable display from onlookers whilst maintaining or improving its accessibility exclusively to the primary user. Examples of the disclosed techniques can manipulate the UI in a way such that it is largely only viewable as intended when the user device is physically configured to a certain folding angle (and / or viewing angle). In some examples, the UI can be customized such that it promotes a certain device configuration (e.g., folding angle) that can provide an optimum security configuration for the user's current surroundings. The UI displayed on the display screen can change in a form factor (e.g., an intended size as viewed by the user) to adapt to the current surroundings and / or on the basis of the sensitivity in the displayed content.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application is entitled "User Interface Manipulation in a Foldable Screen Device" No. 63 / 008,473, filed April 10, 2020, and U.S. Provisional Application No. The application was filed on March 18, 2020, and is entitled "Digital Accessible Phone UI Manipulation for Privacy." This application claims priority to U.S. Provisional Application No. 62 / 991,553, filed on Oct. 13, 2003. each of which is incorporated by reference in its entirety.

[0002] (Technical field) The present disclosure relates generally to data security, and more particularly to mobile end user The sensitive information displayed on the device is then used by onlookers to steal such information. ) to protect against attempts to [Background technology]

[0003] With the proliferation of computers and portable electronic devices, the preferred mode of information presentation has shifted from paper to Generally, such electronic devices provide visual information to a human user. It is equipped with a display screen (e.g., a liquid crystal display (LCD) screen) on which it presents the information.

[0004] In many cases, for example in financial or commercial transactions, Sensitive information such as telephone numbers may be displayed on the display screen. This state of the art is a major obstacle to unscrupulous bystanders. However, it is not possible to view the user's information from the side of the device and to take imaginary or real photos. This creates vulnerabilities that could allow sensitive personal and financial information to be stolen from users. [Brief description of the drawings]

[0005] One or more embodiments of the present disclosure are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings, in which: , where like references refer to like elements.

[0006] [Figure 1A] FIG. 1A illustrates an example environment in which a foldable phone user interface (UI) as presented herein may be implemented.

[0007] [Figure 1B] FIG. 1B illustrates the folding angles of an example foldable phone.

[0008] [Figure 1C] FIG. 1C shows the viewing angle of an example foldable phone relative to a human user.

[0009] [Figure 1D] FIG. 1D shows a table illustrating different configurations of the disclosed UI manipulation techniques.

[0010] [Figure 2A] FIG. 2A shows examples of how a foldable phone UI as presented herein can be implemented with respect to different privacy scenarios.

[0011] [Figure 2B] FIG. 2B shows an example of visual guidance indicators and textual guidance displayed in the actual UI versus the UI as seen by a user.

[0012] [Figure 2C] FIG. 2C shows additional implementation details for some examples of the foldable phone UI manipulation techniques introduced.

[0013] [Diagram 3]FIG. 3 illustrates a functional block diagram of an example UI manipulation engine that can be utilized to implement UI manipulation techniques for a foldable phone.

[0014] [Figure 4A] FIG. 4A illustrates an example data flow diagram for implementing a foldable phone UI manipulation technique with some example components of the UI manipulation engine of FIG.

[0015] [Figure 4B] FIG. 4B illustrates another example data flow diagram for implementing a foldable phone UI manipulation technique with some example components of the UI manipulation engine of FIG.

[0016] [Figure 4C] FIG. 4C illustrates yet another example data flow diagram for implementing a foldable phone UI manipulation technique with some example components of the UI manipulation engine of FIG.

[0017] [Diagram 5] FIG. 5 shows a flow chart illustrating an example method for implementing UI navigation techniques for a foldable phone.

[0018] [Figure 6] FIG. 6 shows a flow chart illustrating another example method for implementing UI navigation techniques for a foldable phone.

[0019] [Figure 7A] FIG. 7A shows a more detailed example of how a distorted UI may be generated in some embodiments of the foldable phone UI manipulation techniques.

[0020] [Figure 7B] FIG. 7B illustrates additional assumptions for the calculations in the example of FIG. 7A.

[0021] [Figure 7C]FIG. 7C shows an example transformation calculation by changing the folding angle in the example of FIG. 7A.

[0022] [Figure 7D] FIG. 7D shows an example transformation calculation due to a change in the viewing angle in the example of FIG. 7A.

[0023] [Figure 7E] FIG. 7E shows an example of how the UI can adjust to compensate for angular distortions (e.g., how tilted) in the actual, generated UI in response to a change in viewing angle.

[0024] [Figure 7F] FIG. 7F illustrates an example relationship between the amount of adjustment of the angular distortion and the change in viewing angle.

[0025] [Figure 8] FIG. 8 shows a flow chart illustrating an example method for implementing a grip manipulation technique.

[0026] [Figure 9] FIG. 9 depicts a flowchart illustrating an example method for implementing a UI-driven grip manipulation technique.

[0027] [Figure 10A] FIG. 10A is an example data flow for implementing a grip manipulation technique using some example components of the UI manipulation engine of FIG.

[0028] [Figure 10B] FIG. 10B is another example data flow for implementing a grip manipulation technique using some example components of the UI manipulation engine of FIG.

[0029] [Figure 11A] FIG. 11A illustrates how individuals in close proximity to a user device may be able to view sensitive information displayed by the user device.

[0030] [Figure 11B] FIG. 11B illustrates how the UI operations engine may calculate which directions should be obscured to ensure privacy, based on the locations of nearby individuals.

[0031] [Figure 11C] FIG. 11C illustrates how the UI navigation engine may result in the display of visual guidance indicia to assist the user in placing their hands in an optimal grip position.

[0032] [Figure 11D] FIG. 11D illustrates how confidential information displayed by a user device can be hidden from nearby individuals when the hands are placed in an optimal grip position.

[0033] [Figure 12] FIG. 12 depicts a high-level block diagram illustrating an example of a mobile system in which at least some operations associated with the techniques presented herein may be implemented. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] References herein to "one embodiment," "one embodiment," and the like refer to the particular embodiment being described. that a feature, function, structure, or characteristic of is included in at least one embodiment of the present disclosure The appearances of such phrases in this specification do not necessarily all refer to the same embodiment. However, referenced embodiments are also not necessarily mutually exclusive.

[0035] As mentioned above, financial and commercial transactions are now carried out using computers and mobile phones instead of paper. Increasingly, such electronic devices are used for mobile communications, such as smartphones, tablets, and other portable electronic devices. A display device is a display screen (e.g., a liquid crystal display (LCD)) that presents visual information to a human user. In many cases, for example in financial or commercial transactions, social security Sensitive information such as phone numbers or bank account numbers may be displayed on the display screen. will take an increasingly larger share of the payments market, and more people will be using their smartphones regularly. People are using smartphones for banking applications. In public places with low security, they may try to enter and read private data. The state of the art is that it is not possible to physically see a user's interactions with their device. and by viewing the user's information from the side of the device, and by visualizing the user's It is a malicious program that can steal sensitive personal and financial information from users by taking photos of them. This creates vulnerability to malicious people and agents.

[0036] Meanwhile, in the past decade, advances in display technology have been primarily focused on organic light-emitting diode (OLED) displays. This allows flexible displays (e.g., using fabric-based substrates) to be There has been great progress in the creation of flexible display devices. Variable screen sizes to accommodate application needs and varying physical configurations Flexible Disk allows for different configurations within a single device, such as allowing multiple As play improves, further refinements will be made to the user interface and modes of interaction. A variety of new physical configurations become possible. Foldable phone UI An element (e.g., a screen(s)) is rapidly manipulated into a different shape, or It can be changed to different folding angles. In addition to flexible displays, traditional displays are also available. Even the PlayStation is becoming more and more popular, with manufacturing costs continuing to fall, and there are more than two PCs with hard displays. Devices with larger screens (e.g. clamshell phones with two screens) are also appearing on the market. is.

[0037] Therefore, what is presented here is the user device's unique IP address, which is used only for the primary user. Flexible, foldable, and easy to access from bystanders while maintaining or improving accessibility to or otherwise obscuring and obscuring confidential information displayed on a reconfigurable display. More specifically, the disclosed user interface (UI) operation technology One or more embodiments of the technology may be implemented such that the user device is physically configured at a certain folding angle. You can manipulate the UI so that you can only see what is primarily intended. In some cases, the UI can provide the optimal security configuration for the user's current surroundings. The devices may be customized to facilitate specific device configurations (e.g., folding angles). The UI displayed on the display screen is adapted to the current environment (e.g., when the user When moving from one environment to another or when the environment around the user changes In addition, or instead, the data (e.g. the intended size as seen by the user) can be changed. Instead, the UI displayed will automatically filter the content based on its sensitivity. You can change the time factor.

[0038] The technology introduced here provides ease of use and readability for users through UI operations. You can hide or obscure information displayed on your device in an intuitive way that maintains your privacy. This allows users to easily create the optimal screen configuration for security. and provide an effective user interface that helps prevent unintentional data loss to bystanders. The face system can be implemented.

[0039] In the following description, folding is used for illustrative purposes only to illustrate various aspects of the technology. The example of a folding mobile device is used. However, the technology presented here is not limited to folding. The present invention is not limited in its applicability to mobile phones or any other particular type of device that can Also, the following description is given with reference to a mobile phone. The focus is on implementing the technology in other electronic devices or systems (e.g., laptops). A foldable mobile phone (top or tablet) may also fit the technology in a similar manner. "Display" or "foldable screen" means a device that is not otherwise clear from the context. Unless otherwise specified, the terms are used interchangeably herein to refer to any display in which the viewing area is foldable. This refers to the deformation of the folding structure, which should include at least two deformations: (1) the folding structure; At least one flexible screen (e.g. one flexible OLE (2) a display having at least two sectors; a display having two sections, each of which has a screen However, the two sections work together (or collectively) as one screen. The two sections are mechanically coupled to each other via a folding mechanism. Some examples of foldable screens are bi-fold displays, tri-fold displays, This includes displays, etc.

[0040] As used herein, the term "user" generally refers to a person who physically operates a mobile device. The term "bystander" generally refers to a person who is in the physical vicinity of a user but does not have access to a mobile device. In addition, the UI operation techniques introduced here are for people who do not physically operate the , simply convert a 1900x1200 pixel display to a 1680x1050 pixel display. By using fewer pixels to display the image, it becomes possible to reduce the size of the screen. ) is distinguished from techniques that simply use less than the entire area to display information, As we will discuss in, UI manipulation techniques require certain Make it intuitive for users to position the phone in a physical device configuration (e.g., folding angle) Note that the visual transformations in this disclosure are used as required. and the term "form factor" in the context of displays generally refers to the The information is provided to the user (i.e., from the user's perspective) and not to a third party (e.g., a bystander). It means how visually larger or smaller something should appear (when viewed from a certain point of view). In this sense, for the purposes of this specification discussion, unless otherwise clear from the context of the specification, the term "Seen" or "perceived" refers to a user of the applicable device and not to another party. It is used with respect to the user's point of view.

[0041] The figures presented herein are merely examples to facilitate review of the disclosed UI manipulation techniques. and note that they are not drawn to scale. [Application environment]

[0042] Figure 1A shows a typical implementation of the user interface for a foldable phone such as the one presented here. 1 illustrates an environment 100 in which a pedestrian may be present. Illustrated in the environment 100 is a large number of bystanders 140. In the presence of a foldable screen 120 on a foldable mobile device 130 A bystander 140 is present in the environment 100 and is therefore The user interface of the mobile device 130 is displayed (i.e., folded in a conventional manner) Bystanders 140 may steal user information (displayed directly on the viewable screen 120). As shown in FIG. 1A, the mobile device 130 Displaying users' private and confidential information where there is a risk to privacy Therefore, the information of the user 110 is protected from being stolen by a bystander 140. It is necessary.

[0043] Thus, as will be discussed in more detail below, the UI manipulation techniques disclosed herein: The device is folded by the user (e.g., at a certain folding angle, and / or , in some embodiments, configured / configured to a selected device configuration (as viewed from a certain viewing angle) Only when reconfigured can the UI be manipulated in such a way that it is effectively visible to the user. In other words, the disclosed UI system operates when the user device is at a particular folding angle and When physically configured for viewing angle and / or viewing position, the resulting UI From a vantage point, the U is visually visible at a size and / or angle that enhances security. You can operate I.

[0044] In one or more examples, the disclosed UI operation system may include a UI navigation system that can be used to navigate around a user's surroundings. By determining the possibility and potential number of Additionally or alternatively, the UI system may determine the level of risk. Determine the level of sensitivity of private and confidential information presented on the mobile device 130 Next, in some embodiments, the level of privacy risk in the user's surrounding environment is and / or private and confidential information presented on the user's mobile device 130. Based on the level of sensitivity of the information, the UI system may allow only the user 110 to prevent distortion of the sensitive information. How to hide confidential information from a foldable screen 1 In these embodiments, the form factor of the display may be determined based on the display screen. The foldable screen 120 can be folded at a selected folding angle and / or at a selected viewing angle. Unless manipulated, the sensitive information may remain distorted even to the user 110. In other words, in some instances, the UI may provide an optimal solution for the user's current environment 100. Specific device configurations (e.g., folding angle and / or viewing angle) that can provide security configurations It can be customized to encourage [Fold angle]

[0045] FIG. 1B illustrates a folding angle 13 of a foldable phone (e.g., user device 130). 8. As shown in FIG. 1B, the fold angle 138 is the multi-section fold. at the angle between sections of a mobile device (e.g., user device 130) that can be More specifically, the folding angle 138 is determined along two axes: A first axis 132 extending along the surface of the first section (e.g., the left section), and and on the surface of a second section (e.g., the right section) of the foldable phone 130. The angle between the second axis 134 extending along the foldable phone 130 is defined by the angle between the second axis 134 extending along the The first and second sections are mechanically connected and foldable along a hinge axis 136. As shown in FIG. 1B, the UI on device 130 is displayed on a foldable display. Collectively (collectively) they are represented by the left and right sections of the ray.

[0046] For ease of discussion, the bi-fold display embodiments described herein are While the present disclosure focuses on a vertical (e.g., left-to-right) folding configuration, the disclosed technology also includes a vertical ( For example, the present invention is equally applicable to those embodiments having a folding configuration (e.g., top to bottom). Note that in some of these embodiments, the configuration may be, for example, a manual or automatic drawing. Through the face rotation feature, the configuration can be adapted to the user's preferences and the current device orientation. They may be interchangeable so that

[0047] According to this embodiment, the folding angle 138 can be selected and the UI will The foldable screen of the foldable phone 130 (e.g., a screen Clean 120) is folded to the selected fold angle (or desired fold angle). When viewed or manipulated, the user interface is optimally readable by the user 110 but is not visible to the surroundings. The selected folding angle is determined by: It can also be expressed as the optimum inner angle at which the display screen 120 is folded. For example, if the display is a bi-fold display (e.g., single hinge), The folding angle is the internal angle around which the display folds.

[0048] In some embodiments, the desired folding angle is displayed in a user interface. Based on the sensitivity of the private information and / or the presence of potential bystanders in the user's vicinity It may be decided to block the user's view of private on-screen information. For example, When a user wants to access private or secure data, Phone camera data may be used to determine the presence of any potential bystanders.

[0049] Once the optimum folding angle for the phone is determined, the user interface displays a message indicating that it is the optimum folding angle. The text can be generated / distorted / customized to be readable at any angle. To make the interface readable at the user's position, the phone needs to be folded and submerged to some extent. It prevents a potential bystander from viewing the user's personal information. For example, a user may I want to see my bank account information while I'm on the train. Software applications installed on the device (e.g., a bank's proprietary applications) When a user accesses the application, the phone presents a distorted user interface to the user. The user is presented with a visual representation of the user interface from the user's point of view. Move the phone narrower until it looks like it is not aligned properly (e.g. edges and fonts do not look askew / distorted). The phone is closed to a narrower angle, and the content is then removed from the user's Block potential bystanders from around the area. [Viewing angle]

[0050] FIG. 1C illustrates an example of a user (e.g., user device 130) of a foldable telephone. For example, the viewing angle 148 for the user 110 is shown. The viewing angle is the angle of orientation of the mobile device 130 relative to the screen. The position of the eye of the user 110 relative to the device 130 may also be understood as the position of the eye of the user 110 relative to the device 130. As shown, in the example of a clamshell phone, the viewing angle 148 is the viewing angle of the unfolded device 130. Between a first axis 142 that is perpendicular to the edge (i.e., 90°) and the line of sight of the user 110 The bi-fold device of FIG. 1C is located in the center (e.g., hinge axis 1). It can be folded around a hinge axis 146 (which has the same configuration as 36).

[0051] As shown in FIG. 1C, for convenience, the disclosure herein will be limited to the focus point (i.e., the point where the user's eyes are Note that we assume that the focus point is the center of the screen. , defines the starting edge from which the viewing angle 148 may be measured. The point assumption may vary depending on the implementation (e.g., the gaze point may be at the top or bottom of the screen). or in some implementations, the gaze point may be dynamically determined by a sensor); The UI operation techniques disclosed herein involve changes in the definition of the viewing angle and / or the position of the gaze point. are equally applicable. [Dynamic vs. static UI adjustments]

[0052] The UI generated by this UI manipulation technique generally has a foldable screen. The device is folded to the intended folding angle and the screen is viewed from the intended viewing angle. Only when the user perceives the image as not visually distorted or distorted (i.e., when the user Thus, in some embodiments, The user device may have the option to adjust the UI distortion dynamically or statically. For the purposes of this discussion, the term "static UI operation" refers to UI operations that are performed at optimal folding angles and and / or the UI is configured to be viewed from an optimal viewing angle (e.g., as described with respect to FIG. 2A below). The device is only manipulated / bent in a certain way, and UI rendering is controlled by the actual folding of the device. in the sense that it does not change based on the viewing angle or the actual location of the user's eye position / gaze point. In contrast, the term "dynamic UI operation" refers to the UI rendering The fold angle of the device and / or the actual eye position / gaze of the user may be affected. This means that the time domain changes and is adjusted based on the position of the

[0053] More specifically, there are four different UI adjustment configurations, as shown in table 150 of FIG. 1D: (A) Both the folding angle UI adjustment and the field of view UI adjustment are static, (B) Static folding (C) Dynamic folding angle UI adjustment and static viewing angle UI adjustment (D) Folding angle UI adjustment and viewing angle UI adjustment are both dynamic There are four combinations: ,The generated UI distortion does not need to take into account the actual folding angle of the device, but rather The UI is generated based on the selected optimal folding angle, and you need to adjust the angle to see the correct UI. Reconfiguring the device to the optimal folding angle (e.g. by folding the screen) Similarly, for embodiments that implement static viewing angle adjustments, the generated The UI distortion does not need to take into account the user's eye position / gaze; rather, the UI must be based on the selected The UI is generated based on the optimal viewing angle, and the user must first view the UI from the optimal viewing angle. Adjust the device to a holding position / orientation / etc. so that the device can be seen from the It is up to the user to move their eyes relative to the device.

[0054] In contrast, for embodiments that implement dynamic folding angle adjustment, the UI distortion is: Dynamically generated and adjusted by taking into account the actual folding angle of the device, i.e. The UI can then morph as the actual folding angle changes. Similarly, dynamic viewing angle adjustment can be implemented. In the embodiment being implemented, the generated UI distortion takes into account the user's eye position / point of gaze. In other words, the UI is dynamically generated and adjusted based on the user's actual eye position / gaze. With dynamic viewing angle adjustment, users do not need to adjust their holding position / orientation / etc. First, the user can generally see the correct UI from any viewing angle available.

[0055] For the sake of brevity, unless otherwise clear from the context, the following descriptions of UI manipulation techniques adopts a configuration of static folding angle UI adjustment and dynamic viewing angle UI adjustment, i.e. , we focus primarily on configuration (B), where configuration (B) is primarily degrees forces the user to change the folding angle, thus protecting sensitive data from bystanders. and secondly, it's easier to change the UI distortion based on where the user's eye position is. The dynamic viewing angle provides convenience and improves usability for the user. It has been found that it may be preferred in many field applications due to at least two advantages: In comparison, configuration (A) does not require any extra sensor system such as a user's eye-tracking system. Low cost and / or power hungry systems that may be expensive and / or consume too much power. Configuration (D) may be preferable for user-friendly and / or low-power implementations. The UI is visible to the user regardless of the current fold angle or gaze point of the It offers the most seamless user experience by far, but the UI looks different. If the ease of use is the same for the user, the user will make some adjustments (e.g., folding the phone This is safer than a configuration with a static folding angle because the user will not want to perform the folding action. It will be lower.

[0056] Although configuration (B) is preferred, the disclosed technology also includes configurations (A), (C), and (D) is also applicable. Specifically, the determination and generation of optimal folding angles is considered. In the following description, the most suitable viewing angle adjustments (e.g., for those implementations with static viewing angle adjustments) will be described. The same or similar considerations (e.g., internal and / or external to the user device) may be used to determine and generate an optimal viewing angle. In some embodiments, the user may apply different configurations. The user may be given the option to select from among the configurations.

[0057] Additionally, in some embodiments, the UI distortion may be used to determine the sensitivity of a user's personal information. "updated" based on the determined level of privacy risk and / or Note that these "updates" of UI distortions are also generally considered to be "dynamic" in some sense. can be understood as a “personal” indicator of sensitivity or surrounding privacy risks. This type of review is often conducted in context because it is conducted in response to changes in either or both of the data. The user's gaze point and / or folding angle (i.e., the user's control This should not be confused with the general consideration of dynamic UI operation in response to a variety of parameters (parameters within the scope of the control). stomach. [User interface operations for protecting confidential data]

[0058] FIG. 2A shows how the user interface of a foldable phone such as the one presented here can be An example of how this can be implemented to protect privacy and confidential data is shown in Figure 2A. Shown is a bi-fold display 220 that folds about a hinge axis 236. Also shown in FIG. 2A are four different examples of a foldable mobile phone 230. Indicative privacy scenarios: (1) private space, (2) relatively safe space, (3 ) public but quiet spaces, and (4) crowded public spaces, due to different folding Four sets of combinations of how the foldable mobile phone 230 can be operated are shown. (i.e., the first set is the example UI 201-202 and the second set is the example U Illustrative UIs 201-208 are UIs 203-204, etc., respectively. So, the privacy scenarios and their corresponding retention angles are just for consideration. Depending on the implementation, more or fewer scenarios may be employed, and different A retention angle may be employed.

[0059] Example UIs 201, 203, 205, and 207 are shown for different privacy scenarios. 2 shows the UI as it appears to a user (e.g., user 110) in the example UIs 202 and 204. , 206 , and 208 show the actual UI displayed on the bi-fold screen 220 . Specifically, the actual UI representations in example UIs 202, 204, 206, and 208 may cause a user's private / confidential information to be transmitted via a bi-fold display (e.g., a bi-fold display) How is the user interface of a telephone having a display 220 laid out? In other words, the "actual UI" representation shows how the foldable phone 230 can How the user interface will appear to the user when fully opened in the scenario. In comparison, the "UI from the user's perspective" representation is Ray 220 was folded at the optimal folding angle for each of the scenarios (and some In some embodiments, when viewed from an optimal viewing angle, the user's private / confidential information How the information appears visually to the user on the user interface of the phone 230 This shows whether

[0060] Take the first privacy scenario - Private Space - as an example. 2 is a diagram showing a state where the bi-fold display 220 of the foldable mobile phone 230 is fully opened. The actual UI rendering of the user’s private / sensitive information on the phone 230 when In comparison, the UI seen by the user, example UI 201, is a bi-fold display 22 0 two sections (i.e. left and right sections) are fully open, i.e. When the actual UI 202 occupies the entire area of ​​the foldable display 220, FIG. 2 shows how the foldable display 220 looks visually to the user. Since it fully utilizes the display area, users generally use it in their private spaces, such as their home. It is understood that some users may prefer to keep the foldable mobile phone 230 fully open. In the fully open state, the actual UI (i.e., example UI 202) is displayed at the user's Note that this is the same UI as seen by the user at the optimal viewing angle (i.e., example UI 201). I want to be.

[0061] The second example privacy scenario - an office cubicle - is generally classified as relatively secure. A place where a user can be seen, but not as secure as the user's home, where bystanders can see the user's phone. Is it not advisable for the user to leave the phone 230 fully open, since there is still a small chance? In this case, keep the phone at a large angle even if it is folded slightly. is sufficient to protect users' private / confidential information from being stolen by bystanders. The example UI 203 is suitable for a scenario in which the user is located in a relatively safe space. 230. Shows actual UI rendering of user's private / sensitive information on the phone 230 In such an environment, the two sections of the bi-fold display 220 are large. When the screen is folded to a small angle (e.g., 135° or more), the optimal view for the user is achieved. In comparison, the UI seen by the user, example UI 203, can be displayed as a bi-fold display. Two sections of I-220 provide the most detailed information on relatively (but not completely) safe spaces. How the actual UI 204 appears to the user when folded to the appropriate folding angle Here, the UI 203 seen by the user is a foldable mobile phone 2 30 is how the actual UI 204 appears to the user when viewed from an optimal viewing angle Note also that the user The UI 203 seen from the outside is the optimal folding angle in a relatively safe space (e.g., U It appears smaller than the full display area (compared to I201), but is distorted to the user. On the other hand, the same user interface may appear distorted to an onlooker. visible (e.g., UI204).

[0062] Similarly, the third privacy scenario, a generally public but still quiet place such as a park, In places that could be classified as safe spaces, especially when users are in more secure locations such as offices, The user folds the phone 230 to a narrower angle compared to the second scenario when the phone was in the It may be a good idea to keep it folded up, compared to a safer place like an office. ,In the park, there is a high possibility that bystanders are looking at the user's phone. Keeping the phone folded at a tight angle exposes the user's private / confidential information. The example UI 206 may be used to protect the user from unauthorized access to the device when the user is in a public place but is still For the scenario where the user is located in a quiet space, In such an environment, the clamshell display is a real UI rendering of the confidential / sensitive information. The two sections of the ray 220 are folded at a narrow angle (e.g., 135°-75°). The UI 205 from the user's perspective can be achieved by folding the The optimal folding angle for a portable mobile phone 230 in a public but still quiet space 2 shows how the actual UI 206 appears to the user when collapsed into a In this way, the display area as perceived by the user (e.g., compared to the UI 201) Even if it is smaller than the entire area of ​​the foldable display 220, optimal folding is possible. At an angle, the UI 205 as seen by the user remains undistorted to the user. However, the same user interface is visible to onlookers due to its narrower folding angle. It becomes more hidden from the public and more distorted to bystanders compared to the second scenario. It appears to be something like that.

[0063] Finally, the fourth privacy scenario, typically in a crowded public space such as a bus or subway. In places that can be classified as in-between, the user is more likely to It may be advisable to fold the phone 230 at a tighter angle. Bystanders are more likely to see a user's phone on the bus than on a train. In this case, folding the Phone 230 at a tighter angle will help protect it from being stolen by bystanders. The example UI 208 can protect the user's private / confidential information. The private / confidential information of the phone 230 user for scenarios when in a public space This is an actual UI rendering. In such an environment, the bi-fold display 220 One section has the narrowest angle (e.g., 75° or less) in all four scenarios. When the UI is collapsed to the , the optimal view for the user can be achieved. 207 is a diagram showing an optimal folding angle for a foldable mobile phone 230 in a crowded public space. 2 shows how the actual UI 208 appears to the user when fully collapsed. As shown in FIG. 2A, the display area perceived by the user is (UI 201, 203, and 2 05) in terms of optimal folding angle, despite being the smallest of all four. Thus, the UI 207 as seen by the user can remain undistorted to the user. However, compared to the first three scenarios, the example UI 208 has the narrowest fold. It is the most hidden from bystanders due to the angle, and the most hidden compared to all previous scenarios. Appears distorted to bystanders. Foldable displays are centered or near the center. In an embodiment configured to be folded (e.g., a bi-fold display configuration) The user interfaces (e.g., UIs 204, 206, and 207) that implement the present technology are 8) is a hinge (e.g., , axis 236).

[0064] In this way, the disclosed UI manipulation / distortion techniques can be used to manipulate the content of a user interface. are visually distorted in the actual UI representation and the display(s) (e.g. , the display 220) is undistorted and readable only when folded at a certain angle This provides an intuitive user interface system that can be viewed as ,An observer may view the actual UI representation (e.g., example UIs 204, 206, and 208) as You can only see distorted user interface displays like those shown in the examples. , protecting the user's private and confidential information from being stolen by an onlooker (e.g., onlooker 140). In addition, the UI representation seen by the user (e.g., the example UI A readable UI, such as those examples shown as I203, 205, and 207, When the user is looking at the screen, the user can fold the display at a specific folding angle. By closing or closing the screen, the bystander cannot see the user's private and confidential information. This will further hinder the theft of information.

[0065] Figure 2B shows the visual guidance as it appears in the actual UI versus how the UI would look to the user. 250 and text guidance 252. As discussed herein, The UI displayed on a foldable or flexible display will be automatically updated based on the selected fold. The UI can be generated based on the angle, so that the foldable phone's screen looks like the It will only be visually visible to the user as normal when folded to the selected folding angle. The UI generated from this technique will at least indicate when the phone is in a selected configuration (e.g., The content will be folded to the selected folding angle) or the content will be abnormal ( This is intuitive to the user because it appears as though the image is oriented in a straight line (e.g., distorted or skewed).

[0066] Nevertheless, it is still possible for a user to intuitively configure and / or orient the phone. and reduce reliance on distortion of displayed content to guide the user. To reduce this, the generated UI shows when the user selects a foldable display. Visual guidance indicators (e.g., guidance indicator 25) that guide the user to adjust the folding angle 0). In some implementations, the visual guidance indicators 250 may include: Appear at the top of the display or in a location that does not obstruct the user's view of normal content in the UI. The visual guidance indicia 250 may be a selected shape such as a circle, a square, a triangle, a diamond, etc. By implementing the disclosed UI manipulation techniques, the visual guidance indicators can be ,The selected display is only displayed when the foldable display is manipulated to the optimal folding angle. As a geometric shape (e.g., a circle as shown in the “User-perspective UI” representation in Figure 2B), For example, if the selected geometric shape is a circle, the fold When the display is not manipulated to the optimal folding angle, the actual UI may take on shapes other than a circle. The user sees the shape of the object as a tilted ellipse (e.g., as shown in the “Actual UI” representation in Figure 2B). In other words, the geometric shapes on the intuitive user interface can be folded. Only when the display is manipulated to its optimal folding angle does it appear as a circle. In some embodiments, the actual UI indicates to the user what the selected geometric shape is. Additional hints on the display (this is part of the text guidance 252 described below) The user then moves the phone closer and closer until the oval appears as a circle. The user interface must be readable from the user's point of view and visually adjustable. The content is perceived as normal, but bystanders around the user may not see that the content would otherwise be blocked. or visually distorted.

[0067] Additionally or alternatively, the generated UI may include the grips discussed with respect to FIGS. 8-11D. A visual guide that guides the user to manipulate their grip according to the manipulation technique. Thus, in one or more embodiments, the generated The UI has visual guidance indicators for UI operation techniques and visual guidance indicators for grip operation techniques. However, the visual guidance indices may be independent of each other. Visual guidance indicators for grip manipulation techniques may be generated from the UI manipulation techniques. independent of (i.e., without visual distortion) or in conjunction with UI manipulation techniques (i.e., may be generated (i.e., with visual distortion).

[0068] Additionally or alternatively, one or more embodiments of the UI operation system herein may be implemented in a manner that is consistent with the current A text guide to guide the user to change the folding angle to the optimal folding angle. Dance (e.g., text guidance 252) may be included on the user interface. For example, text guidance 252 may say, "Open the foldable display more." Or you can tell the user to close the foldable display more. In some variations, the text guidance may be, for example, "Call until the ellipse becomes a perfect circle." The instruction to "close the ) can be displayed.

[0069] In this embodiment, the visual guidance 250 and / or the textual guidance 252 Blocks the view of potential bystanders from the user's personal information and hides the user interface from the user's location. The system asks users to fold the phone just enough to make the screen readable. In a real-world example, a user can provide their bank details while riding a bus. When users access that information on their foldable phones, the phone looks distorted. It presents the user with a UI that allows them to: The instruction is: "Until the black oval becomes a perfect circle (and the UI looks straight and horizontal), "Please fold your phone to a narrower folding angle." The user is Move the phone closer to the screen until it appears as a circle. After adjustment, the user interface The interface is now completely readable from the user's location, but the content on it is hidden. Now blocked by passive bystanders.

[0070] Additionally, in some instances, the UI operation system may prevent sensitive information from being displayed to the user on the UI. A user can require that the user device be placed in an optimal configuration before the UI can be The operating system is best suited to a foldable display that only shows auxiliary UI elements. The user device will not change its configuration unless / until the device is manipulated to a desired configuration (e.g., optimal folding angle). Actively block, blur, mask, or block sensitive data from appearing on device screens For example, the UI may be configured to guide the user Follow the dance instructions and reconfigure the foldable phone to the selected optimal folding angle or until the user operates the device, guidance information (e.g., visual indicators 250 and / or text Only then can the UI operation system display the guidance 252) to the user. Displaying sensitive information in the UI (e.g., in the manner described above).

[0071] FIG. 2C illustrates additional optional implementations of some examples of UI manipulation techniques for foldable phones. As discussed here, at least some of the UI manipulation techniques introduced Some examples are based on the viewing area of ​​a foldable display as perceived by a user (e.g., Decreasing or shrinking UI201 (compared to UI203, 205, and 207) In some embodiments, the content of the display may be folded to have a visual effect. Automatically adjusts based on the determined effective form factor of the available displays. Specifically, in various embodiments, the original layout of the confidential information may be can be repositioned or rearranged into alternative layouts according to the specified form factor, The alternative layout takes up a smaller form factor compared to the total area of ​​the foldable display. It is suitable for the layout factor. For example, the original layout (e.g. it is desktop / tab (which may be designed for larger displays such as red displays) Or an alternative layout designed for mobile devices with smaller screens (e.g. It may use different font sizes, graphics, and / or navigation placement. may be changed to

[0072] Shown in FIG. 2C are two example UIs 270 and 272, where The I270 uses the original layout of the content that should be displayed when the screen is fully open. When a device implementing the UI navigation technology is collapsed, the displayed layout is , may be automatically changed to the alternative layout shown by example UI 272. Or alternatively, additional UI elements, such as scroll bars 274, may be added to the displayed UI. can be added to the UI to navigate content in a simulated reduced screen size. It assists users by providing [Example system components and data flows]

[0073] Figure 3 shows a mobile phone that can be used to implement the UI operation techniques for foldable phones presented here. 3 shows a functional block diagram of an example UI operation engine 300 that can be used.

[0074] The example UI operation engine 300 is a user's mobile device (e.g., As shown in FIG. 3, the UI The operation engine 300 includes a UI sensitivity extraction system 310, an ambient condition assessment system 320, Optimal Display Configuration Finder 330, Actual Display Configuration Finder 340, User The system includes a state evaluation system 350, a UI distortion generator 360, and an auxiliary UI element generator 370. The components shown in FIG. 3 are merely illustrative and may not be implemented in a particular configuration as is well known. Components such as a central processing unit (CPU), memory module, screen display ( Note that the 10 MHz to 10 MHz channels (multiple channels) and communication circuitry are not shown for simplicity. One or more of the systems, subsystems, components, and / or modules presented herein may be The present invention may be implemented as a software application or may include a software application. However, these systems / subsystems / modules Any suitable combination of hardware or firmware to achieve the same or similar functions. Similarly, the example components in FIG. More or fewer elements may be used as shown for ease of review in the specification. Components may be used and may be combined or separated depending on the actual implementation. good.

[0075] According to some implementations, the UI operations engine 300 may be configured to: Receive an indication that classified information will be displayed on a foldable display For example, the application programming interface (AP I) is a step in which the engine 300 receives software as instructions (e.g., from a software application). Implemented for engine 300 to receive software function calls. In a variant, the engine 300 may be provided by an operating system (OS). may be implemented as part of a set of functions provided by the In additional or alternative embodiments, engine 300 may be a stand-alone program that resides in memory. The software can be used to intercept / detect sensitive information displayed.

[0076] After receiving an indication that there is sensitive information to be displayed, the UI operation engine 300 , and to do so, engine 300 takes into account one or more factors, including those based on external and / or internal information. More specifically, According to the UI operation techniques introduced, how much smaller the perceived display screen becomes? Information that may influence the decision of what should be included (i.e., its form factor) is generally There are two categories: (1) information inside the phone, and (2) information outside the phone. The former (i.e., inside information) is extracted by the UI sensitivity extraction system 310. The latter (i.e., external information) can be extracted by the surrounding condition assessment system 320. It is possible. [A.UI Sensitivity Extraction System]

[0077] FIG. 4A illustrates a foldable phone with some exemplary components of the UI manipulation engine of FIG. 4 is an example data flow 400 for implementing UI manipulation techniques for a device. 0 is used to transfer data between various blocks in the UI operation system (e.g., engine 300). Here is an example of how the data can flow. The flow 400 is described herein in conjunction with a functional block diagram of the UI operation engine 300. .

[0078] The UI sensitivity extraction system 310 detects the privacy of data displayed on a user device. It can determine the sensitivity and output the UI sensitivity data. In other words, the UI sensitivity extraction system The system 310 can determine the level of sensitivity of information presented on a user's mobile device. Cut.

[0079] In some instances, UI Sensitivity data may include the appearance of different features currently displayed on the screen. Various sensitivity levels may be described. Examples of high sensitivity items are password entry boxes, bank This may include account information, credit card information, etc. In some implementations, medium sensitivity is This may include personal information such as your social security number, date of birth, home address, or phone number. The sensitivity extraction system 310 extracts public domain information, generic website text, In many embodiments, the user may also identify low sensitivity items such as photos or images. The user may have the option to adjust or edit these items to their own preferences. Additionally or alternatively, the UI sensitivity data may be matched to the highest sensitivity feature on the page. The overall sensitivity rating of the screen may be described.

[0080] According to some embodiments, the UI sensitivity extraction system 310 uses local resources and The UI sensitivity data can be obtained from the UI object and / or from remote resources. Examples of these resources are , secrets about commonly found software applications / components / functions The information may include a look-up table or database in which the degree ratings may be recorded. For example, If the software mobile application is a banking application, If the application window shows a password entry screen, If a web page that provides a password (displaying a detailed description) is triggered or if a UI element If it includes an input field, the resource(s) may be identified as being of high sensitivity. In another or alternative embodiment, the UI sensitivity extraction system 310 may extract metadata, e.g. Message metadata about who the followers are, metadata about the nature of the message , the urgency of the message, and / or the recipient's address (work or personal email In addition, the API provides a way to retrieve UI sensitivity data via In some variations implemented for the UI operations engine 300, a UI sensitivity extraction system The system 310 may receive a UI from the API (e.g., via one or more parameters in a function call). Sensitive data can be received.

[0081] In one or more embodiments, the UI sensitivity extraction system 310 performs a content analysis to In a particular example, the UI sensitivity extraction system 310 may receive the UI sensitivity data. Keywords in sent or received messages (e.g., "highly confidential") based on the presence of a “fidential” identity (e.g., known image processing and recognition algorithms) or By analyzing graphic / photographic content (using rhythm), In some implementations of the UI sensitivity extraction system 310, the level of sensitivity can be determined. The embodiment calculates UI sensitivity by using machine learning techniques (e.g., supervised machine learning). For example, the UI sensitivity extraction system 310 may generate UI sensitivity data. Training data with its sensitivity labelled to establish a decision model The training data can be initially trained with a set of The content (e.g., text, document, image, or video) and their corresponding sensitivity ratings In this manner, the UI sensitivity extraction system 310 can include on-the-fly It can generate a sensitivity assessment of sensitive UI content.

[0082] In addition, some variations of the UI sensitivity extraction system 310 may be implemented to detect changes in the user's mood. UI sensitivity data through user sentiment analysis, e.g. by inferring likely sensitivity For example, when a user operates an application, When a person acts principledly and changes their facial expressions, mood, etc., the UI sensitivity extraction system 310 infers that the sensitivity level of the application is higher than that of general applications These inferential sensitivity level determinations can be implemented using, for example, gesture recognition, facial expression recognition, etc. , and / or may be based on electroencephalography. [B. Surrounding Condition Assessment System]

[0083] The surroundings condition assessment system 320 determines and analyzes the privacy conditions of the user's surroundings. System hardware, firmware, and / or software that may be used to As shown in FIG. 3, the ambient condition assessment system 320 includes a set of hardware components. The ambient condition sensor 322, the ambient condition data preprocessor 324, and the ambient condition fine The present invention includes three exemplary components, da 326.

[0084] More specifically, the ambient condition assessment system 320 may include, for example, an ambient condition sensor 322 Based on the readings, the level of privacy risk in the mobile device’s surrounding environment is calculated. This identified privacy risk level can be used to identify the optimal It can be used by the display configuration finder 320 and the UI distortion generator 360. and the optimal format for user private / confidential information to be displayed on mobile devices. In some embodiments, the performance factor of the image to be displayed on the mobile device is determined. The form factor of the user’s private / sensitive information is based on the identified privacy risk. The level of privacy risk identified can be inversely correlated to the level of The higher the level, the smaller the UI form factor will be seen by the user. For example, As explained above with respect to Figure 2A, there are four levels of privacy risk - There are safe spaces, relatively safe spaces, public but still quiet spaces, and crowded public spaces. The busier the surroundings, the smaller the UI perceived by the user. It would be.

[0085] The ambient condition sensor 324 collects information about the user's surroundings (i.e., "ambient data"). In various examples, the sensor may include one or more sensors that may be used to obtain Edge data may include a visual image or record of a user's location, specific geographic location information (e.g., GPS coordinates), audio information (e.g., background sound data and and / or surrounding audio data), device connection information (e.g. connection data, etc. of the user to detect nearby people's devices) and / or to detect the presence and Applicable peripheral data may include other suitable data that may be used to determine the status. Examples of ambient condition sensors 324 include rear and / or front cameras, or , and other light-based (e.g. infrared) imaging that can generate a light feed from the user's location. The technology includes a GPS sensor that can generate satellite position information, and audio signals collected from the user's location. A microphone or audio sensor capable of generating audio and / or the user's mobile device signatures of devices within the vicinity of the A wireless network transceiver (e.g., Bluetooth®) that can generate In some embodiments, these sensors may include sensors such as It may be included in the mobile device or may be separately connected to the device.

[0086] Specifically, according to one or more embodiments, the ambient condition sensor 322 then receives ambient data can be sent to the ambient condition data pre-processor 324, which can then process the ambient condition data for further analysis. The data can be pre-processed (e.g., to increase the signal-to-noise ratio (SNR) to an appropriate quality). Additionally, the ambient condition data pre-processor 324 may perform the following operations depending on the type of data collected: Various applicable types of pre-processing may be applied to the peripheral data. For example, in some embodiments In this embodiment, the ambient condition data pre-processor 324 detects the light sensor ( Preprocessing optical data generated from a camera (e.g., camera) to estimate scene illumination, focus, vibration, Motion artifacts, reflections, and other features may be corrected. In many embodiments, the ambient condition data pre-processor 324 uses machine vision techniques. to identify key features such as people or faces, and (e.g., to add metadata or tags The optical data can be preprocessed to label it (by adding

[0087] Additionally, some implementations of the ambient condition data pre-processor 324 may filter out background noise (e.g. It can filter out certain audio features (e.g., wind or road noise), amplify certain audio features (e.g., voices), and reduce SNR. The speech data can be preprocessed to further improve the speech quality by increasing R. Similar to the video pre-processing embodiment described above, certain embodiments also include ambient condition data pre-processing. The sensor 324 may record metadata or other information on key features such as clear speech or phrases heard. may preprocess the audio data to add tags.

[0088] Additionally, in certain embodiments, the ambient condition data pre-processor 324 may also include a connection data The data is preprocessed to identify, count, and potentially identify unique device signatures in the user's vicinity. For example, the ambient condition data pre-processor 324 may To identify, count, and / or locate the signatures and / or geographic locations of Information contained in Wi-Fi management frames, requests, beacons, and pings The surrounding condition data pre-processor 324 may, in some examples, utilize GPS. Pre-process satellite data to identify certain qualitative characteristics of the location where the user is currently located (e.g., store name, You can also specify the GPS / satellite / location (location, road name, or landmark). The data will be combined with the phone's existing map services (e.g. Google Maps). Depending on the implementation, preprocessing of the location data may involve (e.g., filtering features) The method may utilize machine learning techniques (to map and classify the images).

[0089] Next, the ambient condition finder 326 extracts the ambient condition data generated by the ambient condition data preprocessor 324. By analyzing the preprocessed surrounding data, the current state of the user's surroundings (i.e. The ambient condition finder 326 can then evaluate the user's surroundings. Based on the neighborhood state, the level of privacy risk in the user’s neighborhood can be determined.1 In the above implementation, the ambient condition finder 326 uses the ambient data to determine the ambient condition. Relevant information can be extracted from the

[0090] In particular, in one or more embodiments, surrounding data may include information about other people (e.g., bystanders) in the vicinity. In some of these examples, the ambient condition finder 326 may include a number In determining the number of other people nearby, the surrounding conditions data preprocessor 324 During processing, a count of the feature tags of different people added to the camera feed can be performed. In this embodiment, the surroundings condition finder 326 uses a machine to count the number of nearby users. By using vision facial recognition technology, the number of other people nearby can be determined. connects to nearby devices detected via Bluetooth technology (or other suitable short-range wireless connection technology). By counting the number of devices, the number of other people in the vicinity can be determined. Alternatively, the ambient condition finder 326 may use fixed conditions identified during pre-processing of the ambient data. By counting the number of human voices present, the number of bystanders can be determined. is a function of the peripheral data based on the change in audio peak amplitude over a given time interval. It can provide a simple estimate of the total sound level in the area.

[0091] Additionally, in some embodiments, the surrounding data may include the spatial distribution and gaze directions of nearby people. In some of these examples, the ambient condition finder 326 may include, for example, Standing to the left or right of the user but probably facing away from the user device (which may be a smaller security threat) Standing in front of the camera and facing the user device (this can be a security threat) This information is detected using camera data and machine vision technology to identify In many implementations, surroundings data is information about known characteristics associated with the user's location, such as For example, how congested a user's location is based on a predicted evaluation of the user's GPS coordinates. For example, the surroundings condition finder 326 may include predicting whether the surroundings are public or not. It can detect that a user is in a public place from the GPS data of the user device, which allows In another example, the ambient condition finder 3 26 can find, through GPS data, that the user is in a field in a large state park, From there, it is inferred that other people are unlikely to be nearby. The ambient condition finder 326 determines the user's location from the user's device's network connection. For example, if a user device is “home” When connecting to "Home Wi-Fi" or "Car Wi-Fi", the ambient condition finder 326 A high privacy level (i.e., a low risk level) may be determined, in contrast ,The user device connects to "public Wi-Fi" or "airport free Wi-Fi" When the surrounding condition finder 326 detects the presence of a high level of privacy (i.e., a high level of risk), It may be possible to decide to set the

[0092] Based on the ambient condition data and through the exemplary method described above, the ambient condition finder 3 26 is the extent to which user data may be exposed when using user devices. In other words, the ambient condition finder 326 can generate an ambient condition output that indicates whether The surrounding state output of the UI operation engine indicates the level of privacy risk in the user's surroundings. In some embodiments, the information may be communicated to other components of the system 300 in a subsequent system configuration. The communication of ambient conditions to the user provides, among other information, a number of risk factors for data exposure to the user. This may be in the form of a concrete description of the people, including their position, gaze angle, and distance from the user. In a variant, the transmission of the peripheral state to subsequent system components is performed at the user's risk (e.g. "Data Exposure Risk"), and The detector 326 performs analysis to determine the likelihood that the user's data may be exposed at the current location. Weighting can be applied to aspects of the analyzed peripheral data. Some additional or alternative examples: In the example, the surroundings condition finder 326 evaluates or ranks the privacy risk of the user's surroundings. Bells can be generated directly. [C. Optimal Display Configuration Finder]

[0093] The optimal display configuration finder 330 is used to prevent other people nearby from observing the display. The optimal physical configuration of a folding or flexible screen (i.e., the “optimal display”) Specifically, the optimal display configuration finder 330 can determine the optimal display configuration for the game. (1) The sensitivity of private and confidential information presented on a user’s mobile device (2) the level of privacy risk in the user's environment; The likelihood that the contents of the screen will be viewed by unscrupulous bystanders is reduced or minimized, and specifies the physical configuration that will be eliminated. In other words, the optimal display configuration fine The UI sensitivity data (e.g., generated by the UI sensitivity extraction system 310 described above) is extracted from the UI sensitivity data. ) and / or ambient conditions (e.g., as determined by ambient condition assessment system 320, also described above). generated by the user to prevent the contents of the screen from being stolen by bystanders. Determine the optimal privacy configuration (e.g., folding angle) for the display.

[0094] Specifically, in some embodiments, the optimal display configuration finder 330 , the internal information of the user device, i.e., the UI sensitivity extraction system 310 Based on the determined level of sensitivity of the user's private information to be In some other embodiments, the optimum display configuration finder 330 can determine is generated by information outside the user device, i.e., the surrounding condition assessment system 320. Based on the determined level of privacy risk, the optimal folding angle can be determined. Additionally or alternatively, the optimal display configuration finder 330 may be based on the user's preferences. Determined level of sensitivity of private information and determined level of privacy risk The optimal folding angle can be determined based on both the static viewing angle adjustment and the In this embodiment, the optimal display configuration finder 330 further determines the optimal viewing angle. can.

[0095] Next, some embodiments of the optimal display configuration finder 330 find the optimal folding angle, i.e., the optimal internal angle at which the display is folded (FIG. 1B and (See related discussion above.) If the user device's display is a single joint (e.g. In those embodiments including a single hinged display or multiple displays, The optimal display configuration can be the optimal folding angle at which the display is folded. For example, acute folding angles can reduce the perceived size of the display. may provide higher display privacy (relative to the high privacy risk area) Ideal for large areas and / or highly sensitive content, obtuse folding angles reduce Offers a larger perceived size while still providing less viewing privacy It is well suited for low privacy risk areas and / or low sensitivity content. do.

[0096] The optimal display configuration can vary depending on the physical characteristics of the user device. For example, In some embodiments, the device comprises two or more folding arms designed with specific pre-defined folding configurations. The device may have a folding section. In such a device, the folding of the screen The viewing angle and rotation direction may be limited, so that the optimal display configuration is For folded sections, the combination of the fold angle plus a vector for a specific rotation direction Additionally, in certain instances, the device may be a flexible display. Such a device may be a device having a predetermined folding fabric. A single piece folding slide with dynamically configurable folding behavior that is not limited to a single action. In such devices, the folding angle and rotation of the screen In this way, the optimal display configuration is determined by the particular selected The configuration may include, for example, the direction in which the screen should be folded and the and the location of the center of the fold (e.g., fold line, etc.) on the surface of the fold. The combination of the folding angle and the folding line is determined by, for example, the auxiliary UI element generating unit 370 ( (described above) to fold the display fabric into a selected shape. In this context, many examples of fold lines are foldable ( For example, a bi-folding display can be made with the same hinge axis as that introduced here. can.

[0097] In some cases, the sensor is based on an accelerometer / gyroscope sensor built into the user device. We use this data to identify how you are currently holding your device and Through this, the optimal display configuration finder 330 determines the optimal folding angle for the device. It remains to be determined whether this can be physically accomplished by folding one or both sides. I want to be loved.

[0098] After the optimal display configuration is determined, it is passed to the optimal display configuration finder 330. A purely static UI manipulation embodiment (i.e., the UI does not handle the actual folding) can be output by (which do not change with changes in angle or user's eye position), e.g. For configuration (A), the optimal folding angle and / or viewing angle are determined based on UI rendering and and output to the UI distortion generator 360 for display. [D. Actual Display Configuration Finder]

[0099] FIG. 4B illustrates a collapsible UI using some exemplary components of the UI manipulation engine of FIG. 4 is an example data flow 402 for implementing a UI manipulation technique for a mobile phone. Row 402 is a block diagram illustrating a flow of operations between various blocks within a UI operation system (e.g., engine 300). An example of how data can flow is shown in Fig. 3 and Fig. 4B. 1, the data flow 402 is a functional block diagram of the UI operation engine 300. These are also explained next.

[0100] As previously introduced, many embodiments of the disclosed UI manipulation techniques include dynamic UI manipulation, That is, when the user's actual gaze point changes (e.g., in configuration (B)), or When the actual folding angle of the user device changes (e.g., in configuration (C)), or both (e.g., in configuration (D)), where UI distortion changes or distorts These embodiments rely on one or more of the above as the basis for these dynamic UI adjustments. system components (e.g., the real display configuration finder 340, or the user state The rating system 350) is introduced below.

[0101] The real display configuration finder 340 finds the current configuration (or "real For example, the actual display configuration finder 340 can determine the actual display configuration. is device data obtained from one or more display configuration sensors built into the device. These display configuration sensors can be used to determine the actual display configuration. Examples of information that can be obtained from the This can include the location and / or orientation of the device (e.g., tilt angle). In some cases, the device orientation is determined by the accelerometer, gyroscope, or the user device. The data may be obtained from other suitable on-board sensors. It may take into account how the user is currently holding the device and / or how to achieve the optimal viewing angle. How should a user move their mobile device to achieve the desired effect (e.g., viewing angle adjustment is required to achieve a stable For embodiments that are of interest,

[0102] In some embodiments, the real display configuration file is used to collect device data. The display configuration sensors used by the aligner 340 may include mechanical sensors. For example, a two-panel (e.g., two-section or bi-fold) flexible display In play, the mechanical sensor may be built into the hinge that joins the two panels. In one example, the sensor is a flexible electronic material that changes resistance when the hinge closes or opens. so that it can measure the folding angle of the screen. The angle data tells us how the foldable screen is currently folded and / or Or how users adjust the foldable screen to achieve the optimal folding angle. This can be used to determine whether the fold angle adjustment should be statically reconfigured. (For embodiments of the present invention.)

[0103] Generally speaking, flexible displays (e.g., fabric displays) Some embodiments of the actual display configuration describe the surface distortion across the surface. In order to achieve this, the distortion of the display at multiple points across the surface of the display must be sufficiently high. For example, in many of these embodiments, the actual display configuration may include For each point on the surface, data representing a vector field with scalar magnitude and angle of distortion is It may include data.

[0104] Additionally, some variations of the display configuration sensor may also include one or more cameras. For example, typically in flexible display embodiments, a camera (which is a device may be built into the device or obtained from a third-party device The optical data obtained can be used by the real display configuration finder 340 to Create a normal map or vector field for the current device surface. In an alternative embodiment, the camera(s) may also be configured to view the actual display configuration using a real-time display viewfinder 340. can be used to determine the relative device orientation and spatial location of the device. Furthermore, in some embodiments, the real display configuration finder 340 estimates ,It can use camera data and image processing technology to ,capture the actual display configuration in 3D, It can result in a 3D model file, e.g. an STL file.

[0105] In various implementations, the display sensor detects the movement of light to the object and the object. Calculates the distance to an object using the time it takes for the light to be reflected back from the object. It can include a Time of Flight (ToF) sensor to measure. For example, a built-in ToF sensor can ,The device by performing a measurement of the distance between the device and a person or other physical feature. This can be used by the real display configuration finder 340 to obtain the position. Alternatively, the display configuration sensor may include a radar. or other object or motion sensing technologies may be used between display elements or on the The real display configuration file is used to determine the spatial relationship between the play elements and the user. The signal may be used by the datum 340. [E. User Status Evaluation System]

[0106] The user condition assessment system 350 is used to determine and analyze the physical condition of the user. The user state evaluation system 350 may be a combination of system components such as: Exemplary components include a user status sensor 352 and a user status finder 354. This can be done.

[0107] The user status sensor 352 obtains data regarding the user's status (or "user status"). The user condition assessment system 350 may include one or more sensors that can be used to assess the user condition. More specifically, an example of a user state that can be acquired by the user state sensor 352 is The user's gaze point, which is the position on the screen where the focus is, and the user's position relative to the display. head angle, which is the angle of the user's head; viewing angle, which is the angle of the user's eyes relative to the display; and / or head orientation, which is the orientation of the user's entire body relative to the display. In some implementations, the user state sensor 352 may be a dedicated, eye-tracking sensor system. or a combination of sensors on the user device. Exemplary components of the user condition sensor 352 include, for example, one or more front Cameras, motion sensors (which may be on the user device or in a smart band) a suitable projector and / or time-of-flight sensor. For example, a projector can project a particular light or light pattern (e.g., , infrared or near infrared) into the user's eye, and the camera can capture the user's eye and the pattern. Images can be captured, after which image processing and machine vision techniques can determine the user's eye position and In another example, a time-of-flight sensor may be applied to determine the screen and the gaze point. The distance of the user from the target can be detected. [F.UI Distortion Generation Section]

[0108] The UI distortion generator 360 generates visual distortions in the user interface (herein, “UI UI distortions are also called UI transformations or UI manipulations. From the user's current viewpoint at the optimal display configuration (e.g., optimal folding angle) Manipulate the visibility of the displayed UI to get the best view of the user interface. The visibility of the display from any other bystander's point of view is The image is reduced (due to visual distortion) and obscured (due to visual distortion), making it difficult for bystanders to see the image. This reduces the possibility of stealing a user's personal information from the user's device.

[0109] Depending on the implementation, UI distortions can include a combination of different transformations and distortions. . For example, UI distortion is a set of transformations (e.g., translations) applied to a user interface. , rotation, or tilt). According to various embodiments of the present invention, the UI distortion generator 36 0 means the UI and target information on the UI can only be read from the desired angle and / or distance The user interface can be used to express optical illusions (e.g., the one discussed in Figure 2A). ) Specific implementations with "holographic display" hardware In this embodiment, the UI distortion generator 360 may, for example, determine the angle at which the pixel light of different pixels is reflected. It can generate a set of instructions that includes how privacy-sensitive UI elements should be adjusted, 2A. As mentioned above, the type and degree of distortion should be determined based on the level of sensitivity of the information presented. Any transformations or distortions applied to high sensitivity UI content must be considered. To make it much less readable for non-target users (e.g. UI examples UI example 207, compared to 203), can be more extreme.

[0110] In a particular example, the UI distortion generator 360 may take into account a user state data when determining and generating the UI distortion. Considers the actual display configuration and / or user state data. For example, by using the user's gaze point, the UI distortion generator calculates the movement of the user's eye position. Adjusted the viewing angle of UI distortion so that it remains visible to the user. Using a real display configuration, many embodiments of the UI distortion generator 360 can be It also allows for continuous correction of UI distortion in response to any change in folding angle. and the perceived UI form factor (e.g., size) that is generated through UI distortion. ) in response to new actual display configurations and / or user conditions (e.g., The information can be updated and adjusted (in real-time or near real-time manner).

[0111] Additionally, some embodiments of the UI distortion generator 360 may compensate for the actual display configuration. In some implementations, the device hardware can generate instructions to compensate for the The I-distortion generator 360 compensates for visual degradation caused by viewing the screen at the outer edge of the viewing cone, for example. In order to make the UI element visible, we use commands to increase the brightness of pixels in certain parts of the UI element. In another variation, the UI distortion generator 360 may generate a new art in the new UI. Aspect ratios, UI positioning, UI element distortion, etc. so gestures still work For example, the device may generate instructions for recalibrating the 3D control gestures of the device. .

[0112] Further, for improved user experience and usability, the UI distortion generator 360 Some embodiments can modify one or more characteristics of the UI. For example, the UI distortion generator 36 0 may modify the dimensions of user interface features. The distortion generator 360 can modify the positioning of UI elements on the display. The distortion generator 360 adjusts the distortion of the display to reduce the amount of distortion required for extreme folding. It may move elements from the edge to the center. In some embodiments, the UI distortion generator 3 60 allows modification of many user interface features including, for example, color, brightness, or pixel distribution. In some examples, the UI distortion generator 360 may perform the steps described above with respect to FIG. When the device (such as the example) is closed to a narrower folding angle, the display becomes smaller. It allows you to rearrange the UI to a layout that is more appropriate for your application (i.e., as perceived by the user). do. [G. Auxiliary UI element generation section]

[0113] FIG. 4C illustrates a collapsible UI using some exemplary components of the UI manipulation engine of FIG. 4 is an example data flow 404 for implementing UI manipulation techniques for a mobile phone. Row 404 is a block diagram illustrating a flow of operations between various blocks within a UI operation system (e.g., engine 300). An example of how data can flow is shown in Fig. 3 and Fig. 4C. 1, the data flow 404 is a functional block diagram of the UI operation engine 300. These are also explained next.

[0114] The auxiliary UI element generator 370 provides a set of UI elements to guide the user in achieving an optimal display configuration. UI elements or features that can be used to More specifically, in some instances, The assistant UI element generating unit 370 allows the user to operate the folding screen correctly and to display the optimal display. The characteristics of the auxiliary UI elements are presented to the user when the ray configuration (e.g., optimal folding angle) is achieved. (e.g., UI distortion from UI distortion generator 360 and / or uses optimal display configuration parameters from the optimal display configuration finder 330 You can choose to display one or more auxiliary UI elements (for example,

[0115] More specifically, in some embodiments, the auxiliary UI elements may have distortion effects on the UI itself. and / or additional visual guidance indicators (e.g., selected geometric shapes) It may include dimensional changes to intuitive shapes or patterns. UI elements are distorted and only become perfectly rounded when the screen is changed to the optimal display configuration. In some examples, the selection may include a circle (or an ellipse, e.g., indicator 250 in FIG. 2B). Additional textual guidance to the user to tell them what the selected geometric shape is (e.g., a circle). In order to provide more useful assistance to the user, In one or more embodiments, this additional text guidance is provided to the device when the device is in an optimal display mode. Without being configured (e.g. folded) into a certain configuration (e.g. at an optimal folding angle) or before), providing that the image is displayed in an orientation that the user can see - such An example of this is shown as text guidance 252 in FIG. In some implementations, it is determined by the actual display configuration (e.g., the current folding angle) and the The display may be generated based on the difference between the desired display configuration (e.g., the selected folding angle) and the For example, the text guidance may suggest changing the folding angle, i.e., opening more. In some cases, instead of a geometric shape, the user can be prompted to Images may be used.

[0116] In an additional embodiment, the auxiliary UI elements are displayed when the display is in an optimal display configuration. The embodiment may include a hardware enablement function such as an LED indicator when may include a holographic display indicator, which may can display the correct color or brightness when in the optimal display configuration. Alternatively, auxiliary UI elements may assume a specific shape once the display is in its optimal display configuration. Or it may include a pattern of pixels, or points of illumination, that would appear only as an image. For example, in a fabric display, the fabric is manipulated to the optimal shape. Only when the pixels are machine-readable, one-dimensional or two-dimensional barcodes (e.g., QR codes (registered (trademark)), thereby confirming that the optimal display shape has been achieved. It functions as:

[0117] Further, in some variations, the auxiliary UI element generator 370 displays only the auxiliary UI elements. and a foldable display that is capable of facilitating an optimal device configuration (e.g., optimal folding angle Displaying sensitive data on the screen of a user device unless / until it is operated by a third party You can actively block, blur, mask, or otherwise impede, or simply prohibit, For example, the auxiliary UI element generating unit 370 may block a UI that displays confidential materials. In addition, the UI distortion generator 360 is controlled to provide guidance information (e.g., visual indicator 250 , text guidance 252 (in FIG. 2B), and / or the LED indicators described above. The user is then shown only the instruction and the flip phone. Only when the selected optimal folding angle is changed, the UI operation engine 300 (e.g. , showing confidential information on the UI (in the manner described above with respect to FIG. 2A). [Methodology]

[0118] FIG. 5 illustrates an example method 500 for implementing UI navigation techniques for a foldable phone. 5 shows a flowchart of a method for displaying confidential information on a foldable UI display. To protect private information and data from bystanders, user devices (e.g. For example, due to UI distortion and rendering on the device 130, 230 (FIGS. 1A, 2A) The various components (e.g., configurations) of a UI manipulation engine (e.g., engine 300; FIG. 3) The method 500 may be implemented by any of elements 310-370 (FIG. 3). This is introduced with simultaneous reference to Figures 2C and 3.

[0119] First, the UI operation engine operates the display of a mobile device (e.g., device 230). The user may receive instructions to display confidential information on a display (e.g., display 220). 510). One example of such an instruction may be a software function call. In an example, the mobile device may include a foldable display. The UI operation engine optimizes how sensitive information should be displayed. In various embodiments, the UI operations engine may first determine the user's configuration (520). identify (522) the level of sensitivity of the information to be displayed on the user's screen, and then Determine the form factor for how sensitive information should be displayed. Or, alternatively, the UI manipulation engine determines the form factor for the perceived UI. When determining whether a device is suitable for use, it is important to consider the privacy risks (or how crowded) surroundings the user device are. The form factor, as described above, can generally be determined as follows: Perceived display size, i.e., after UI distortion techniques, the user The foldable display can be folded to the optimal folding angle to optimize the device. The size of the display as seen by the user when it is manipulated (e.g., folded) into a display configuration In general, the more sensitive the data displayed, or the more sensitive the surrounding environment. The higher the privacy risk, the greater the perceived form factor of the displayed UI (e.g. The size (size) of the device is smaller. That is, the form factor is more resistant to environmental risks and / or can be inversely correlated to the sensitivity of the data displayed. According to the above embodiment, the determined form factor may be, for example, For privacy scenarios, it will be smaller than the full area of ​​the foldable display .

[0120] The UI manipulation engine then creates a foldable A UI may be generated (530) to be displayed on the display. For example, the UI manipulation engine may The actual phone configuration can be received (532), and the UI displayed by the UI manipulation engine can be Display sensitive information in a form factor (e.g., perceived size) but not in a foldable The possible displays depend on the selected physical phone configuration (e.g., selected folding angle and and / or in some embodiments a selected viewing angle), As shown in the example UI of FIG. 2A, the collapsible In an embodiment of the display, the UI may be displayed across multiple sections of a foldable display. In a particular implementation, the foldable display may be bi-folded. It can be used as a display.

[0121] In particular, in one or more embodiments with a foldable display, a mobile device Unless the display section of the device is folded to the folding angle, the user interface UI distortions are generated so that the interface appears distorted to the user. After the folding angle is determined, the display section of the mobile device will adjust to the folding angle. Unless the user interface is collapsed into a In the example, the UI is transformed symmetrically around the axis along the hinge of the foldable display. (e.g., similar to those UI examples 204, 206, and 208). The UI distortion engine receives information about the motion of the device, for example from one or more orientation sensors on the mobile device. The current orientation state of the mobile device can be received, and the received current orientation state of the mobile device can be The UI can be further adjusted in response to the current orientation conditions.

[0122] Further, in some embodiments, the UI manipulation engine may, for example, determine the user's gaze point. Based on the user state, the UI operation engine can , a mobile device (e.g., in those embodiments that perform dynamic viewing angle adjustment) Unless the user interface is oriented at the viewing angle to the user, the user interface will be distorted. In some of these examples, the UI manipulation engine can determine the viewing angle of the user. The position of the user's eyes can be determined and the UI can be adjusted in response to the detected position of the user's eyes. In our implementation, the UI interaction engine employs a user-facing eye detector on the mobile device. and a new optimal viewing angle for the UI reflects the detected position of the user's eyes. , you can further tweak the UI.

[0123] FIG. 6 illustrates another example method 600 for implementing UI navigation techniques for a foldable phone. 6 is a flowchart illustrating a method 600 for displaying confidential information on a foldable UI display. ,In order to protect private information and data from ,bystanders, the user device ( For example, updating the UI distortion and rendering on the device 130, 230 (FIGS. 1A, 2A). Various components (e.g., components of a UI manipulation engine (e.g., engine 300; FIG. 3) for updating For example, the method 600 may be implemented by components 310-370 (FIG. 3). will be introduced with simultaneous reference to Figures 2A-2C and Figure 3.

[0124] The UI operations engine receives updates about the actual current physical configuration of the user device (e.g. Once the current fold angle ( 610 ) is received or retrieved, the UI distortion update can be performed. More specifically, in some embodiments, the UI operation engine may include the following new information: It can detect one or more of the following: It can detect new indications of confidential data being displayed (6 12), which introduces a new level of privacy risk in the environment surrounding user devices. The system can detect (614) new user states (e.g., new gaze points or user (616) can detect the new actual data (the new eye position of the user) and / or The new device configuration (e.g., new actual folding angle or new device orientation) can be detected. (618).

[0125] Next, consistent with the aspects described above (e.g., with respect to FIG. 3), the UI operations engine In some examples, a security feature for the display may be determined (620). When there is a new or updated level of data (612) and / or When there is a new level of privacy risk (614), the UI operation engine Optimal device configuration (e.g., new optimal folding angles and / or some implementations) In this case, the new optimal viewing angle can be determined (620). In addition, dynamic viewing angle adjustment is implemented. For those that do, when a new user state (e.g., a new gaze point) is detected, (616) The UI manipulation engine creates a new optimal device configuration for UI distortion generation ( For example, a new viewing angle (620) needs to be determined. For those that implement the adjustment, the new actual device configuration detected (e.g., the new When there is a folding angle (at the time of folding) (618), the UI manipulation engine uses A new optimal device configuration (e.g., new folding angle) must be determined (620). do.

[0126] The UI operation engine then adjusts the UI according to the new, updated optimal device configuration. In one or more implementations, the UI distortion updates occur as the user manipulates the foldable display. In one or more embodiments, steps 610-620 are performed substantially in real time as the device creates the image. The selection step of 630 may be performed iteratively or recursively. [Example]

[0127] FIG. 7A illustrates a foldable screen device (e.g., device 130, FIG. 1A; 2A) of a bi-fold display (e.g., display 120, FIG. 1A 220 (FIG. 2A), a distorted UI is generated (e.g., by UI distortion generator 360; FIG. 3). More specifically, (e.g., as described above with respect to FIG. 3) The UI form factor for how sensitive information should be displayed was determined. Then, based on the determined form factor, it will be displayed on the foldable display. The actual UI that should be displayed is generated. To this end, the following is provided to calculate the UI transformation given the above input parameters in Figure 7A. 1 is an equation and description of an exemplary method for calculating

[0128] As shown in FIG. 7A, the mobile device 730 has a central axis 736 (e.g., a hinge). The bi-foldable display 720 is folded around the center. The distance (X1) from the axis 736 to the edge of the bi-foldable display 720 is known. Furthermore, assume that the distance (X2) from the central axis 736 to the user's eyes is (for example, , an eye tracker and / or a ToF sensor, which may be mounted on the device 730 Assume that the presence of a hazard can be detected (based on Finally, in the example shown in FIG. 7A, the folding angle is represented by (α) and the viewing angle is (β).

[0129] Let the length of the edge result of the manipulated UI be (Y1), and the length of the center result of the manipulated UI be (Y2). Let the length be (Y2) (as shown in Figure 7A). The initial length of the edge of the original undistorted UI Let the length be (Y0) (as shown in FIG. 7B). The relationship between Y0, Y1, and Y2 is as follows: It can be represented by the following transformation:

number

number

[0130] FIG. 7C shows an example of a transformation calculation by changing the folding angle α in the example of FIG. 7A. Specifically, the following shows an example calculation method for T1.

[0131] Continuing to refer to FIGS. 7C and 7A, assuming the viewing angle β is constant, Point P1 (at the center of the display, where the central axis or hinge is) is It is assumed that the thickness of the device is negligible compared to ΔX2. Assume that the transformation T1 is from the viewpoint of an observer at a distance X2 (at the edge of the display) is applied to compensate for the increase in the perceived change in height of the UI feature at point P2 (at It is possible.

[0132] In this coordinate system, the center of the display, the folding axis (P1) (e.g. axis 736) Since the size of the UI feature remains unchanged, Y2 = Y0, and T1 is on the central axis. It does not need to be applied to any UI features that are

[0133] Determine the UI feature at point P2, which is a distance X1 away from the origin (e.g., at initial height Y0). To do this, the transformation T1 reduces the perceived size of the feature by an amount ΔX2 that causes it to be closer to the observer. Therefore, a transformation can be applied to obtain the smaller height Y1. Such a transformation can be applied to the feature at P1 (assuming height Y0) to obtain It can be expressed as follows.

number

number

[0134] Thus, based on the coordinate system of this example of FIG. 7C, transformation T1 can be expressed as follows: can be done.

number

[0135] FIG. 7D shows an example of a conversion calculation according to a change in the viewing angle β in the example of FIG. 7A. The following shows an example of the calculation method for T2.

[0136] With continued reference to FIG. 7D and FIG. 7A, assuming the folding angle α is constant: Point P1 in FIG. 7D (at the center of the display, where the gaze point is) is the original plane of the device. It is assumed that the thickness of the device is negligible compared to ΔX2. Then, the transformation T2 is from the viewpoint of the observer at distance X2 (at the edge of the display) may be applied to compensate for the increased perceived change in height of the UI feature at point P2 (which is do.

[0137] The viewing angle β in three-dimensional (3D) space has three components: the xy plane, the yz plane, and and one each for rotation in the zx plane. Each can be considered independently. However, similar equations can be applied for the calculation.

[0138] In the above coordinate system, Y2=Y0 and T2 is applied to the UI feature at the center. The size of the UI feature at the central axis (P1) remains unchanged so that it does not need to be be.

[0139] For a UI feature at point P2, a distance X1 away from the origin (e.g., at initial height Y0), , the transformation T2 compensates for the perceived increase in size due to the feature being closer to the observer by an amount ΔX2. Thus, a transformation can be applied to obtain the smaller height Y1 by multiplying P1 by (assuming height Y0).

number

number

[0140] Therefore, based on the example coordinate system of FIG. 7D, the transform component T 2xy is expressed as follows: Can be done

number

[0141] A similar transformation can be done for the other components of β: yz and β zx may also be applied to the transformation components T 2yz and T 2zx These then (using equation (2)) represent the transformation T2 Used for:

[0142] FIG. 7E shows the angular distortion (e.g., An example of how the UI can adjust to compensate for the angle (e.g. how tilted it is). First, FIG. 7E shows a display configuration (e.g., the best display configuration finder 3 30, how the UI elements (as determined by Figure 3) appear visually to the user 7 is an example UI 701 that shows how to set the display to a flat screen. It can be displayed in a section, for example, in the foldable display discussed herein. It can be one section of the display sections that together form a For ease of discussion, the following discussion focuses on adjusting the angular distortion of one section. Although the technology focuses on a single point, it could be applied to multiple sections of a foldable display as well. The UI 701 is displayed on a yz plane, with the x-axis extending into the page, as shown in FIG. As shown, the UI 701 is represented by a round dot and is visually This includes corners that would essentially form a right angle (e.g., 90°). This angle is used to illustrate how an example angular distortion adjustment is performed in response to changes in the viewing angle β. Assume that the angle that this angle forms is (γ).

[0143] Further shown in FIG. 7E are example UIs 702, 703, 704, and 705. The example UIs 702-703 are configured to display sections that are viewed at acute angles by the user. β (i.e., less than 90°), and UI704-705 are for the display The section is viewed by the user from an obtuse viewing angle β (i.e., greater than 90°). Here, without compensation for the change in viewing angle, the example UI 702 has As shown, when the display section is viewed from an obtuse viewing angle β, the angular angle It is noted that γ is an acute angle (as opposed to a right angle for UI701). Similarly, Without compensation for the change, the display section When the image is viewed from an acute viewing angle β, the angle γ is (as opposed to a right angle for UI701) ) becomes obtuse.

[0144] Therefore, at least some embodiments (e.g., the dynamic viewing angle UI adjustment described above) According to the implementation, the UI measures the perceived change in tilt angle due to the user's change in viewing angle. Angular distortion adjustments (e.g., at the corners of box UI elements, skewness) to account for The view angle UI adjustment can be performed by adjusting the view angle of the example UIs 703 and 70 5, the angular distortion caused by the change in the viewing angle can be compensated. Some embodiments disclosed herein may be used to determine the presence or absence of these features in the UI based on changes in the viewing angle β. A mechanism can be employed to determine how much the tilt angle of the needs to be adjusted.

[0145] Specifically, in some examples, from the user's perspective, the angle γ of the angle is expressed as a hyperbolic tangent function ( This varies with the viewing angle β based on the That is, in certain embodiments, the difference between the displayed angle γ and the viewing angle β can be The relationship may be described, for example, based on a tanh() formula.

number

[0146] FIG. 7F illustrates an example of the relationship between the adjustment amount of the angular distortion and the change in the viewing angle, more specifically, the formula ( 9) As shown in Figure 7F, the angle on the UI is inversely proportional to tanh(β). When the viewing angle β is 90°, the displayed angle on the actual UI can be increased or decreased. The angle γ is also 90°, i.e., when the viewing angle β is vertical, no adjustment of the angle γ is required. Please note that:

[0147] In a practical implementation, one or more of the parameters in equation (9) (i.e., “a”, “b”, “c”) , and / or "d") may be based on the actual device design and application environment, e.g. display characteristics, display size, and / or sensor input (where the user's eyes are However, it should be noted that the method may be optimized or adjusted based on the location of the object (e.g., how far away and / or where the object is located). Please note that in the embodiment shown in FIG. 7F, a=1.5, b=2, c=π, and d=1 In some embodiments, the parameters are based on user habits and personal characteristics of different people. (e.g. holding angle or distance) can be actively adjusted for each user to better suit them.

[0148] It should also be noted that the tanh() function described above is merely an example and in one or more embodiments It is noted that functions other than the tanh() function may be used to achieve a similar effect. For example, one variation uses a "sigmoid" function (whose plot resembles "tanh"). Additionally, some examples may use linear or curves with different slopes. You can use a combination of multiple lines to approximate the visual effect described here. A function is (1) a function that crosses the origin at 90° (i.e., when β is 90°, γ is also 90°). (2) it is recognized that the function can be characterized by a plot that is symmetric about the origin. In some embodiments, it is (3) an increase in β leads to a decrease in γ, and vice versa. It is further characterized by the fact that β and γ are inversely correlated. In some embodiments, when β is near 0°, γ is near 180°, Conversely, when β is near 180°, γ is near 0°.

[0149] In the above-mentioned embodiment, the disclosed UI operation technique is to Flexible, foldable, or otherwise, while maintaining or improving accessibility Hiding or obscuring sensitive information displayed on a reconfigurable display from bystanders This way, loss of user data is prevented and the usability and readability of the UI is improved. In addition to maintaining the same performance, device configurations can be dynamically customized to suit the needs of the user. It provides optimal security configuration for the user's current environment. In addition, it provides auxiliary UI elements It is an intuitive interface that can help users easily achieve the optimal screen configuration for security. A UI system may be implemented.

[0150] Here, in at least some embodiments, the UI manipulation techniques presented are Transforms ( For example, one or more existing software developments that can help execute It can be implemented based on the kit (SDK). In general, the UI technology introduced can be built The transformation class can be a subcategory under the SDK related to geometric image transformations. These can be called perspective transformations. Available functions for perspective transformations An example of such an SDK is OpenCV TM In these SDKs , the dimensions of the input image (e.g., the original undistorted edge of the UI such as Y0 in Fig. 7B) input describing the perspective reference (e.g., view angle, or object plane), and There can be many functions that can receive the desired output and the input. Often the functions are "Straighten" the images on two planes and present them on a plane facing the observer. Any of the embodiments of the present disclosure may be utilized, programmed, or otherwise instructed to perform the functions described herein. In some embodiments, the desired output is determined by taking into account changes in viewing angle and / or folding angle. It may be from a "pre-distorted" graphic. [Grip operation to protect confidential data]

[0151] As mentioned above, one approach to obscuring sensitive information from nearby bystanders is the folding angle. A visual guidance indicator is provided on a foldable device of the user device to prompt the user to operate the The visual guidance indicators are to be displayed on the foldable display. Only when the ray is manipulated to the selected folding angle does it fit the selected geometry. The visual guidance indicators may be designed to be visible as a fold. When the display is not operated at the optimum folding angle, it may be folded to a different shape (e.g., tilted). It may appear as an oval.

[0152] Another approach to blocking sensitive information from nearby bystanders is to block the view of nearby bystanders using a user-blocking technique. Visual guidance indicators that prompt the user to adopt a grip position on the device A proper grip position reduces the risk of data loss. The risk of data loss may be determined based on the optimal placement of hands and fingers for The determined angle at which the observer is present around the user device and therefore the display. This may be based on what parts are currently likely to be visible to those bystanders.

[0153] For example, the selected geometric shape (e.g., a circle) is related to the selection of the selected hand (e.g., the left hand). The user device then displays the selected geometric shape along with instructions to place a selected finger (e.g., index finger) on the selected geometric shape. In such an embodiment, the selection may be displayed on a foldable display of the device. The position of the selected geometry is determined by the folding angle and the position of nearby bystander(s). ) The embodiment may be based on a user device having a foldable display. Although the feature may be described in the context of a flexible or otherwise reconfigurable device, The present invention is equally applicable to user devices having a display. The process of protecting confidential information is flexible and flexible to prevent loss of confidential information in a seemingly natural way. or by a user device having a reconfigurable display that is not foldable. This may be done.

[0154] As discussed further below, potential security and Data relating to privacy risks and / or displayed by user devices Data regarding the sensitivity of information is used to determine the optimal grip to avoid information loss to nearby individuals. The optimal grip configuration can be used to determine the user's grip position. This can be used to generate UI element(s) that encourage or assist the user. For example, The current grip position data is used to guide the user to move their grip to the optimal grip position. This can be used to determine which UI element(s) should be generated. The grip location may be, for example, a touch-sensitive element, a pressure-sensitive element, a proximity sensor, etc. , may be determined based on readings generated by an ambient light sensor or the like.

[0155] FIG. 8 shows a flow chart illustrating an example method 800 for implementing a grip manipulation technique. The method 800 is for protecting sensitive information presented by a user device from nearby individuals. To this end, various components (e.g., At a high level, the method 800 may be implemented by: The method that would best prevent unintentional disclosure of sensitive information displayed by a user device. It can be described as an algorithmic approach to determining pose.

[0156] First, the UI operation engine displays the sensitive information on the display of the user device. An example of such an instruction may be a software function call. As described above, the user device may include a foldable display, a flex display, The UI may include a swivel display or a display that is otherwise reconfigurable. The manipulation engine determines the optimal display configuration ( An optimal grip position (also called an "optimal grip position") can be determined (820). The UI operation engine identifies the level of sensitivity of the information to be displayed (822) and then Determine the form factor for how the confidential information is displayed. Additionally, or Alternatively, the UI manipulation engine may take into account the user's inputs and outputs when determining the form factor for the UI. The privacy risk in the device's surroundings can be determined (824). The higher the level of sensitivity of the data presented or the higher the privacy risk in the surrounding environment. The smaller the UI form factor, the smaller the UI form factor. The risk of fraud may be inversely related to the risk and / or sensitivity of the information.

[0157] Additionally, the UI manipulation engine determines the optimal grip configuration to obscure sensitive information. The optimal grip configuration (also called the “optimal grip position”) can be determined by the peripheral condition. 3 , as determined by a surrounding condition finder (e.g., surrounding condition finder 326; FIG. 3 ). For example, the ambient condition finder may be based on the visibility of the display. The goal is to block the line of sight of these individuals to prevent machine vision Facial recognition technology may be used to determine the number of nearby individuals. If is flexible or collapsible, the UI manipulation engine will use its current shape or The optimal grip position may further be determined based on the folding angle. The operation engine may use data about the surrounding environment and / or information sensitivity, To determine the privacy requirements of a play and to obscure any sensitive information presented therein; Identify the optimal grip position.

[0158] In some embodiments, the optimal grip position is determined based on the physical dimensions and capabilities of the user. The hand is further informed by the characteristics of the hand (commonly referred to as "hand characteristics"). Examples of hand characteristics are the hand, fingers, and This includes palm size (measured, for example, in terms of width and length) and flexibility. These hand characteristics may be manually entered by the user, or these hand characteristics may be automatically generated by the mobile It may be determined algorithmically based on past interactions with the mobile device, or These hand characteristics are estimated for a given user based on demographic averages. This estimate may be used directly by the algorithm or the average may be calculated as: Further refinements may be made using the approaches described above.

[0159] As mentioned above, the optimal grip position is determined by the distance between the display and the user's hand. More specifically, the UI operation engine may determine the location of the object based on (e.g., Optimal grip positions for blocking the line of sight of these nearby individuals (as determined from edge data) The display may be positioned (e.g., as determined from UI sensitivity data). As an example, a UI manipulation engine might The viewing angle of an individual may be determined and then which viewing pathways are selected to protect sensitive information. The UI operations engine may then determine whether any of the following paths need to be blocked: (i) Blocking the visual pathway; and (ii) being feasible taking into account the characteristics of the user's hand. One or more grip patterns may be generated.

[0160] Next, the UI operation engine determines, based on the optimal display configuration and the optimal grip configuration, A UI may be generated for display on the display (840). For example, the UI operation engine may: Data regarding the actual configuration of the user device and the location(s) of nearby individual(s) and then receiving (842) a notification from the at least one display on which the sensitive information is presented. MAY display a UI element (or elements) that indicates where the user should place their hands to hide parts of the screen. Thus, depending on the nature of the display, the UI manipulation engine can adjust the orientation, folding angle, etc. As shown in the example UI of FIG. 10, data indicating the degree or shape of the hand may be obtained. 1. Indicates where selected fingers should be placed to ensure optimal grip position The above UI elements may be shown.

[0161] FIG. 9 is a flow diagram illustrating an example method 900 for implementing a UI-driven grip manipulation technique. 9 shows a chart of a method 900 for collecting sensitive information displayed by a user device from nearby individuals. To protect against this, various configuration elements of a UI manipulation engine (e.g., engine 300; FIG. 3) may be The present invention may be implemented by any of the components (e.g., components 310-370; FIG. 3).

[0162] The UI operations engine receives updates about the actual current physical configuration of the user device. Once the UI is updated (910), the UI manipulation engine can perform an update. For example, the UI manipulation engine can It may detect one or more of the following: (912) This is a new privacy risk in the environment surrounding the user device. The system can detect a new level of the user state (914), which can be a new user state (e.g., a new grid The new actual device configuration (916) can be detected and / or the new actual device configuration (917) can be detected and / or the new actual device configuration (918) can be detected and / or (e.g., new orientations, fold angles, or shapes) can be detected (918).

[0163] As an example, the UI operation engine may be configured to detect one or more sensors (e.g., The data generated by the grip position sensor (called "grip data") The grip position can be continuously monitored by checking the grip position sensor. Examples include touch screen support components (e.g., touch sensitive elements and pressure sensitive elements). These include sensors such as a sensor, proximity sensor, ambient light sensor, gyroscope, and accelerometer. In some embodiments, grip position sensing is achieved by vibration (e.g., by a motor or piezoelectric element). In addition, or alternatively, this may be assisted by active generation of stimuli, e.g. For example, an optical sensor (e.g., a 3D printer) facing the user may be used to determine hand position based on the corneal reflex. Grip data may be collected from the user device and its The position of the palm relative to the display, and the user device and its display Various features of the grip position may be described, including the position of the thumb(s) relative to the thumb. Therefore, the UI manipulation engine uses the grip position generated by the grip position sensor(s). Grip data may be obtained and then the grip data is examined to determine the optimal grip position. This is also fine.

[0164] Next, consistent with the approach described above (e.g., with respect to FIG. 8), the UI manipulation engine Then, a new optimal grip position can be determined (920). For example, the UI manipulation engine can If there is a new or updated level of classified information for display (912), and / or or if there is a new level of privacy risk in the surrounding environment (914), As an example, the UI manipulation engine may determine an optimal grip position when the surrounding environment becomes more crowded. If you decide to make the UI more cluttered (and therefore less private), The engine requires a new optimal grip position that will further obscure classified information. It may be determined that

[0165] The UI manipulation engine can then adjust the UI according to the new optimal grip configuration (9 30). As discussed further below with respect to FIG. 11, the adjustments are made based on the finger (or fingers) selected by the user. In one or more embodiments, the method may include adding a UI element(s) that indicates where the user interface should place the In this case, UI adaptation is the process of adjusting the display as the user interacts with it (e.g., its orientation, folding, etc.). This is done virtually in real time (by changing the folding angle or shape). Similarly, UI adaptation works in a virtually realistic way as the user moves their hand relative to the user device. For example, the location of the confidential information may be changed by interacting with the display ( For example, the position may be changed in response to a determination that the user has moved their hand to touch. Steps 910-930 may be performed iteratively or recursively.

[0166] FIG. 10A illustrates a grip using some example components of the UI manipulation engine of FIG. 1 is an example data flow 1000 for implementing a manipulation technique. How data is transferred between various blocks in the UI operation system (e.g., engine 300) 4A. Data flow 1000 is an example of how data can flow. This is substantially similar to Law 400. However, here, the characteristics of the user's hand are taken into account. The data for the optimum display configuration finder 330 is also obtained. These data determine which grip positions can be feasibly achieved by the user. It can be used to

[0167] 10B illustrates a graphical representation of a UI operation engine using some exemplary components of the UI operation engine of FIG. 1 is another example data flow 1002 for implementing a lip manipulation technique. Grip position sensor(s) 910 generate grip data from which hand position can be inferred. An example of a grip position sensor is a touch screen support component (e.g. touch-sensitive and pressure-sensitive elements), proximity sensors, and ambient light sensors , gyroscopes, accelerometers, etc. Additionally or alternatively, for example, corneal reflex The system includes an optical sensor (e.g., a camera) facing the user to determine hand position based on the Using the grip data, the grip position finder 920 may: The actual grip position of the user can be determined. In other words, the grip position fine The analyzer 920 analyzes the grip data generated by the grip position sensor(s) 910. Based on the analysis, the actual position of the hands can be determined. Information about the actual grip position can be used to The UI elements may be provided to the UI element generator 370, which may be used to find the optimal display configuration. 330 and the optimal display configuration determined by the grip position finder 920. Based on the determined actual grip position, a UI requirement is generated to implement the grip manipulation technique. The method may be configured to generate a factor (multiple factors).

[0168] 11A-D show an example implementation of the grip manipulation technique. First, the user device 11 A UI manipulation engine (e.g., engine 300; FIG. 3) running on 00 may be configured as shown in FIGs. 8-9. As discussed above with respect to the location of nearby individuals 1102a-c, such an application may Roach has determined that these individuals cannot view the display of the user device 1100. Allows UI operations to identify visual pathways that must be blocked in order to More specifically, the UI operation engine performs a privacy-ensuring process as shown in FIG. It is then possible to calculate which direction(s) must be blocked in order to achieve this.

[0169] The UI manipulation engine then selects the appropriate UI element(s) for presentation on the display. ) can be determined. For example, a UI manipulation engine can determine the It can be determined where the visual guidance indicator should be shown on the display. For example, the UI operation engine may display a visual guidance indicator 1104 in the upper right corner of the display. In some embodiments, the visual guidance indicator 1104 causes the presentation of One of multiple visual guidance indicators shown on a display. For example, a UI The maneuvering engine may determine that multiple visual guidance indicators are needed in a crowded environment. Multiple visual guidance indicators accommodate selecting fingers on the same or different hands. You may do so.

[0170] A visual guidance indicia 1104 is provided for positioning a selected finger on the visual guidance indicia. When the user places the selected finger 1106 over the visual guidance indicia 1104, If placed in the vicinity, the view of nearby individuals 1102a-c would be obstructed (thereby exposing confidential information). In some embodiments, visual guidance is provided to ensure that the information remains private. The indicia 1104 is designed to be substantially covered by a selected finger 1106. In this example, a selected finger 1106 can see part of the display while being visually guided. The image completely covers the performance indicator 1104.

[0171] Many user devices allow for multi-touch functionality (or simply "multi-touch"). Multi-touch is a technology that allows a touch-sensitive display to simultaneously It allows the user to recognize the presence of multiple points touching the display. The device 1100 allows one finger to be held in one area while another finger touches another area. When touching a screen, multiple contact points may be detected. To take this into account, Part of the play may occur outside the multi-touch target area in some embodiments. For example, the user device 1100 may have a touch-sensitive display. Touches that occur within a fixed area of ​​the ray (e.g., the visual guidance indicator 1104 and its surroundings) The user device 1100 may be instructed not to detect the match event, or the user device 1100 may be instructed not to detect the match event. touch events to determine multi-touch behavior. Therefore, touch-sensitive displays can handle multi-touch events. At least one part is recognized for touch purposes and the touch event is multi-touch and at least one portion that is not recognized for the purpose, and a visual guidance indicator ( or multiple visual guidance indicators) indicate that a touch event is recognized for multi-touch purposes. The information may be placed in one or more of those portions that are not recognized.

[0172] The visual guidance indicia 1104 also guides the selected finger 1106 to the touch-sensitive display. Note that it may represent instructions for placement completely outside the bounds of the splay. As shown in FIG. 11C, the visual guidance indicator 1104 indicates that the selected finger 1106 is 1100. It can be done.

[0173] At a high level, the UI element(s) generated by a UI manipulation engine generally: It serves one of two purposes. First, the user must enter the You need to interact with these UI elements (or elements) to achieve the optimal grip position. Second, the user may manipulate these UI element(s) to perform a desired function. Examples of desired functionality may include verifying a payment, entering a password, and viewing financial or personal information. Therefore, the user may be The user may not be able to perform the desired function.

[0174] In some embodiments, the UI element(s) are based on the existing UI elements required for a given function. For example, the UI operation engine may define a UI element called "Show password". Existing graphics that have been labeled may be used, but their characteristics (e.g., position and The UI (and size) may be adjusted to meet the requirements described above. The element(s) represent new elements that meet the requirements above. For example, a UI manipulation engine The fingerprint(s) indicating where the selected finger(s) should be placed is sent to the designated function. It may be overlaid on the UI where the function can be completed.

[0175] Each UI element created by the UI manipulation engine determines its appearance, function, and position. Examples of such properties are dimensions, color, animation, and These properties, in turn, cause the corresponding UI elements to be activated. You can specify the action, if any, required to In some embodiments, these properties are applied to UI elements ( are influenced by fundamental characteristics of the underlying UI on which the object (or objects) is created / selected. For example, The color of the UI element(s) will be changed to match the color scheme of the underlying UI. obtain.

[0176] In some embodiments, aspects of the grip manipulation technique may include: For example, the user may use Specify where nearby individuals 1102a-c are located (by tapping the As another example, the user may be prompted to It may be possible to influence the location, number, or placement of the sensor indices 1104. For example, the user is first prompted to place a selected finger on the visual guidance indicia 1104. However, the user must perform some action (e.g., tap at least twice and then drag) ) may be performed to change the position of the visual guidance indicia 1104. Thus, the characteristics of the user's hand may affect the visual guidance indicia shown on the user device 1100. In some embodiments, the user may use the visual Different arrangements of the sensory guidance indices are shown on the display of the user device 1100. The user may be prompted to complete a calibration process. The user may select a different number of visual guidance indicators located at each location. based on the speed and ease with which the selected finger can be positioned on one or more visual guidance provided by the The UI manipulation engine may learn the most appropriate arrangement for the user. [Computer System and Device Architecture]

[0177] FIG. 12 illustrates a computing device on which at least some of the operations described herein may be implemented. 12 is a block diagram showing an example of an operating system 1200. For example, a computing device (e.g., , the components utilized to implement the user devices 130, 230 of FIGS. 1A and 2A 12 illustrates several components of a computing system 1200.

[0178] The computing system 1200 includes one or more processors communicatively coupled to a bus 1216. A central processing unit (also called a "processor") 1202 on the A volatile memory 1210, a network adapter 1212 (e.g., a network interface interface), video display 1218, input / output device 1220, control device 1222 (e.g. A drive unit including a recording medium 1226, e.g., a keyboard and a pointing device 1224, and a signal generator 1230. The bus 1216 may include appropriate bridges, adapters, One or more physical buses and / or points connected by an adapter or controller 12 is illustrated as an abstract representation of a two-point connection. is a system bus, a PCI (Peripheral Component Interconnect) bus or a PCI- Express bus, HyperTransport or ISA (industry standard architecture) bus, SCSI (Small Computer System Interface) bus, USB (Universal Serial Bus) versal serial Bus), IIC (I2C) bus, or IEEE (Institute of Electrical and Electronics Engineers (formerly known as "Firewire") standard 1394 bus (also called)

[0179] The computing system 1200 may be a personal computer, a tablet computer, or the like. computers, mobile phones, game consoles, music players, wearable electronic devices (e.g. monitors or fitness trackers), network-connected ("smart") devices (e.g. TV or home assistant devices), virtual / augmented reality systems (e.g. headsets, mounted display), or by the computing system 1200 Another method that can execute (sequentially or otherwise) a set of instructions that specify an action or actions. may share a similar computer processor architecture as that of other electronic devices. stomach.

[0180] Main memory 1206, non-volatile memory 1210, and recording medium 1226 ("machine readable medium"). Although the term "machine-readable medium" is used herein to refer to a single medium, the term "machine-readable medium" is used herein to refer to a single medium. " and "recording medium" refers to a single medium or multiple media that store one or more sets of instructions 1228. a centralized or distributed database and / or associated caches and servers In addition, the terms "machine-readable medium" and "recording medium" are intended to include 1200. " " shall be construed to include any medium capable of carrying or transmitting information.

[0181] In general, the routines executed to implement the embodiments of the present disclosure include operating A system or a specific application, component, program, object, module, A computer program is a program that is written in a computer program code or sequence of instructions (collectively called a "computer program"). A computer program may be implemented as a programmable unit. One or more instructions (e.g., For example, instructions 1204, 1208, 1228 are included. When read and executed, the instruction(s) cause the computing system 1200 to , causing operations to be performed to implement elements relating to various aspects of the present disclosure.

[0182] Additionally, the embodiments are described in the context of a fully functional computing device. However, those skilled in the art will appreciate that various embodiments may be distributed as program products in a variety of forms. It will be understood that this disclosure is not intended to be construed in a limiting sense, but rather to be illustrative, not restrictive, and not restrictive. applies regardless of the particular type of computer readable medium.

[0183] Further examples of machine-readable recording media, machine-readable media, or computer-readable media include volatile volatile and non-volatile memory devices 1210, such as floppy disks and other Removable disks, hard disk drives, optical disks (e.g. compact discs Screed-only memory (CD-ROM), digital versatile disc (DVD), etc. recordable type media, as well as transmission types such as digital and analog communication links. Includes media.

[0184] The network adapter 1212 enables the computing system 1200 to Any communication protocol supported by the operating system 1200 and an external entity. Through the protocol, the computing system 1200 communicates with entities that are external to the computing system 1200 on a network. The network adapter 121 enables data transfer within the network 1214. 2, network adapter cards, wireless network interface cards, routers, Access points, wireless routers, switches, multi-layer switches, protocol converters, Gateways, bridges, bridge routers, hubs, digital media receivers, and / or Or it may include a repeater.

[0185] The network adapter 1212 provides access to data in a computer network. Determine and / or manage proxy permissions and manage access across different machines and / or applications. The system may also include a firewall that tracks different trust levels between applications. The rule applies to machine and application, machine and machine, and / or application specific It is possible to enforce a given set of access rights between a given set of entities (e.g. Hardware and / or software that regulates the flow of traffic and resource sharing between There can be any number of modules with any combination of software components. The firewall further restricts the access of objects by individuals, machines, and / or applications. An access and operation right that details the permissions, including the user's access and operation rights, and the circumstances under which the permissions apply. The user may manage and / or have access to an access control list.

[0186] The techniques presented herein involve the use of programmable circuitry (e.g., one or more microprocessors) ), software and / or firmware, special purpose hardwired (i.e. It may be implemented by any suitable (i.e. non-programmable) circuitry, or a combination of such forms. The special purpose circuitry may consist of one or more application specific integrated circuits (ASICs), programmable logic devices, Programmable logic devices (PLDs), field programmable gate arrays (FPGAs), etc. It could be. [remarks]

[0187] The foregoing description of various embodiments of the claimed subject matter has been presented for purposes of illustration and description. It is not a matter of being exhaustive or presenting the claimed subject matter in the precise form disclosed. It is not intended to be limiting. Many modifications and variations will be apparent to those of ordinary skill in the art. The embodiments have been chosen and described in order to best explain the principles of the invention and its practical application. , thereby allowing one of ordinary skill in the relevant art to easily understand the claimed subject matter, various embodiments, and contemplated aspects. They will be able to understand various modifications that are suitable for the particular application for which they are designed.

[0188] Although the detailed description describes certain embodiments and the best mode contemplated, No matter how detailed the description may appear, the technology can be practiced in many ways. While encompassed by the present disclosure, the details of their implementation may vary considerably. A particular term used when describing a particular feature or aspect is The present invention is not limited to any particular feature, characteristic, or aspect of the technology described herein. Generally, the following claims should not be construed as meaning that The terms used are those disclosed herein that are intended to be understood as meaning the same unless those terms are expressly defined herein. It should not be construed as limiting the technology to the particular embodiments shown. The actual scope of the technology may vary from the disclosed embodiments to any embodiment of the invention. All equivalent methods are also included.

[0189] The language used in this specification has been chosen primarily for ease of reading and explanation. It is not chosen to define or enclose a subject. The scope of the present technology is not determined by this detailed description, but rather by the disclosure of the application based thereon. It is intended that the disclosure of various embodiments be limited by any claims that follow. The following disclosure is intended to illustrate, but not to limit, the scope of the technology defined in the following claims. It is not intended to be something that is. [Industrial Applicability]

[0190] This disclosure is applicable to data security.

Claims

[Claim 1] 1. A method for privately displaying information, the method comprising: receiving an instruction to display confidential information on a mobile device, the mobile device includes a foldable display; determining the form factor for how the confidential information is displayed; Thus, the form factor is smaller than the total area of ​​the foldable display. , to determine, Based on the determined form factor, displaying on the foldable display. generating a user interface for displaying the user interface; The sensitive information is displayed in the form factor, and the foldable display is selected. generating a selected angle that is visually transformed to a user unless manipulated; Detecting the position of the user's eyes from an eye detector on the mobile device facing the user. and adjusting the user interface in response to the detected position of the user's eyes. To adjust, A method for providing the above.

Images