UI display methods, devices, head-mounted display devices, and storage media

Abstract

This application relates to the field of UI display technology, and provides a UI display method, apparatus, head-mounted display device, and storage medium. The method includes: acquiring first real-time information about a user's current use of the head-mounted display device; querying historical information about the user's use of the head-mounted display device to determine a target operation matching the first real-time information; querying the historical information to determine the user's gaze point frequently used operation area; and displaying the UI corresponding to the target operation in the gaze point frequently used operation area. Embodiments of this application enhance the flexibility of UI display.

Description

Technical Field

[0001] This application relates to the field of UI display technology, and in particular to a UI display method, device, head-mounted display device, and storage medium. Background Technology

[0002] AR (Augmented Reality) glasses can project virtual images into the user's field of vision, achieving perspective and fusion effects, and are applied in various scenarios such as navigation, gaming, education, and healthcare. Currently, the UI (User Interface) layout of AR glasses is fixed, presenting users with a monotonous and rigid interface.

[0003] Therefore, how to enhance the flexibility of UI display has become an urgent problem to be solved. Summary of the Invention

[0004] This application provides a UI display method, apparatus, head-mounted display device, and storage medium, aiming to enhance the flexibility of UI display and improve the user experience.

[0005] To achieve the above objectives, this application provides a UI display method, the UI display method comprising:

[0006] Obtain first-hand real-time information about the user's current head-mounted display device;

[0007] Query the user's historical usage history of the head-mounted display device to determine the target operation that matches the first real-time information;

[0008] Query the historical record information to determine the user's frequently used gaze area;

[0009] The UI corresponding to the target operation is displayed in the frequently used area of ​​the gaze point.

[0010] In addition, to achieve the above objectives, this application also provides a UI display device, which includes a memory and a processor;

[0011] The memory is used to store computer programs;

[0012] The processor is configured to execute the computer program and, when executing the computer program, implement the steps of the UI display method as described above.

[0013] In addition, to achieve the above objectives, this application also provides a head-mounted display device, which includes the UI display device as described above.

[0014] In addition, to achieve the above objectives, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the UI display method described above.

[0015] This application discloses a UI display method, apparatus, head-mounted display device, and storage medium. By generating historical information of a user's use of the head-mounted display device, when the user is currently using the head-mounted display device, the method obtains the first real-time information of the user's current use of the head-mounted display device, then queries the historical information to determine the target operation that matches the first real-time information, and determines the user's gaze point frequently used operation area. Then, the UI corresponding to the target operation is displayed in the determined gaze point frequently used operation area. This enables the UI layout display of the head-mounted display device to be set according to the user's behavior, rather than being fixed and singular. Therefore, the flexibility of the UI display is enhanced, thereby improving the user experience. Attached Figure Description

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 This is a schematic flowchart illustrating the steps of a UI display method provided in an embodiment of this application;

[0018] Figure 2 This is a schematic diagram of a user's eye fixation point and the position information of the user's gaze point provided in an embodiment of this application;

[0019] Figure 3 This is a schematic diagram of the gaze point location information of multiple user gazes provided in an embodiment of this application;

[0020] Figure 4 This is a schematic diagram of a gaze point region provided in an embodiment of this application;

[0021] Figure 5 This is a schematic flowchart illustrating the steps for determining the user's gaze point's frequently used operating area, as provided in an embodiment of this application.

[0022] Figure 6 This is a schematic diagram of a first type of location information and a second type of location information provided in an embodiment of this application;

[0023] Figure 7 This is a schematic diagram of a commonly used gaze point operating region provided in an embodiment of this application;

[0024] Figure 8 This is a schematic diagram of a process for controlling the UI display of AR glasses according to an embodiment of this application;

[0025] Figure 9 This is a schematic diagram illustrating an application icon display provided in an embodiment of this application;

[0026] Figure 10 This is a schematic block diagram of a UI display device provided in an embodiment of this application. Detailed Implementation

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0029] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0030] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0031] Embodiments of this application provide a UI display method, apparatus, head-mounted display device, and storage medium for enhancing the flexibility of UI display and improving user experience.

[0032] Please see Figure 1 , Figure 1 This is a flowchart illustrating a UI display method provided in one embodiment of this application. This method can be applied to head-mounted display devices, as well as various types of electronic devices such as smartphones, tablets, desktop computers, and personal digital assistants. The application scenarios for this method are not limited in this application. Head-mounted display devices include, but are not limited to, AR glasses, VR (Virtual Reality) glasses, and other similar devices.

[0033] like Figure 1 As shown, the UI display method specifically includes steps S101 to S104.

[0034] S101. Obtain the first real-time information of the head-mounted display device currently being used by the user.

[0035] For example, taking AR glasses as an example, when a user is currently using the AR glasses, relevant real-time information is obtained. For ease of distinction, this real-time information will be referred to as the first real-time information below. For instance, the first real-time information includes at least one of the time the user is currently using the AR glasses and their geographical location. Similarly, for ease of distinction, the time in the first real-time information will be referred to as the first moment, and the geographical location in the first real-time information will be referred to as the first geographical location. For example, the first moment of current AR glasses use is T1, and the first geographical location is A1.

[0036] In some embodiments, a UI personalization display function switch control is pre-configured, allowing users to choose to turn the switch control on or off as needed, thereby further enhancing the user experience. When the user turns off the switch control, the UI is displayed in the normal manner. When the user turns on the switch control, the UI personalization display function is enabled, and relevant first-real-time information is obtained when the user is currently using AR glasses.

[0037] In some embodiments, before obtaining the first real-time information of the user's current use of the head-mounted display device, the method includes: recording the user's gaze point position information and relevant element information corresponding to the user's operation each time the user uses the head-mounted display device; and generating the historical record information based on the relevant element information and the gaze point position information.

[0038] Before the user uses the AR glasses, the system records the user's gaze point location information during each past use of the AR glasses. For example, the AR glasses have a built-in eye-tracking device, which includes, but is not limited to, an eye tracker. The eye-tracking device collects spatial distribution data of the user's eye gaze points to obtain the corresponding gaze point location information. For example, such as... Figure 2 As shown, the user's eye fixation point in a given instance is as follows: Figure 2 As shown in 2(a), the corresponding user gaze point position information is recorded as follows: Figure 2 As shown in 2(b) above. After the user has used the AR glasses for a period of time, the gaze point location information of the user's gaze is recorded multiple times, for example, as shown in... Figure 3 As shown in the figure, the multiple red dots represent the gaze points of the user's gaze over multiple viewing points.

[0039] In addition, each time a user performs a corresponding user action, relevant information related to that action is simultaneously recorded. This relevant information includes, but is not limited to, the user action itself and the real-time information surrounding its execution. For example, each time a user performs a specific user action, the system records the specific action performed, the time and location of the action, and increments the cumulative number of times the action has been performed (i.e., counts the cumulative number of times the user action has been performed). For instance, if a user opens the Didi Chuxing app, the system simultaneously records the time and location of opening the app and increments the cumulative number of times the action has been performed.

[0040] For ease of distinction, the real-time information corresponding to the user's operation will be referred to as the second real-time information below. The second real-time information includes at least one of the following: the time when the user performs the user's operation and the geographical location. For ease of distinction, the time in the second real-time information will be referred to as the second moment, and the geographical location in the second real-time information will be referred to as the second geographical location.

[0041] Based on the relevant element information recorded when the user performs multiple user operations, as well as the recorded gaze point position information, corresponding historical record information is generated. The historical record information includes, but is not limited to, the second time point, the second geographical location, the cumulative number of times the user operation is performed, and the gaze point position information of the user's gaze.

[0042] S102. Query the user's historical usage history of the head-mounted display device to determine the target operation that matches the first real-time information.

[0043] After obtaining the first real-time information of the user's current use of AR glasses, the system queries historical records based on this information to identify user actions that match the first real-time information and designates these matching user actions as target actions. For example, after obtaining the first moment when the user is using AR glasses as T1 and the first geographical location as A1, the system queries historical records based on the first moment T1 and the first geographical location A1 to identify target actions that match the first moment T1 and the first geographical location A1.

[0044] In some embodiments, querying the user's historical data on the head-mounted display device to determine the target operation matching the first real-time information includes: querying whether there is second real-time information matching the first real-time information in the historical data; if so, determining the user operation corresponding to the matching second real-time information as the target operation.

[0045] By querying historical records, the system compares the first real-time information of the user's current use of AR glasses with the second real-time information in the historical records to determine if there is a match between the second and first real-time information. If no second real-time information matches the first real-time information, no response is processed; otherwise, if a second real-time information matches the first real-time information, the user action corresponding to the matching second real-time information is identified as the target action.

[0046] In some embodiments, querying whether there is second real-time information matching the first real-time information in the historical record information includes: if the time difference between the second time and the first time in the historical record information is less than a preset time threshold, and / or if the distance between the second geographical location and the first geographical location in the historical record information is less than a preset distance threshold, then it is determined that there is second real-time information matching the first real-time information in the historical record information.

[0047] For example, a preset time threshold can be set to 30 minutes, and a preset distance threshold can be set to 1 kilometer. It should be noted that the specific values ​​of the preset time threshold and preset distance threshold can be flexibly set according to the actual situation, and no specific restrictions are imposed in this application.

[0048] For example, taking the example listed above, if the preset time threshold is 30 minutes and the preset distance threshold is 1 kilometer, and the first moment when the user is currently using the AR glasses is T1 and the first geographical location is A1, by querying the historical information, it is determined that there is a second moment T2 in the historical information where the time difference between the second moment T2 and the first moment T1 is less than 30 minutes, and there is a second geographical location A2 where the distance between the first geographical location A1 is less than 1 kilometer. Then it is determined that there is a second real-time information in the historical information that matches the first real-time information. In the matching second real-time information, the second moment is T2 and the second geographical location is A2.

[0049] Next, the user actions corresponding to the second time point T2 and the second geographical location A2 in the historical records are identified as the target actions. For example, if the user performed the user action of opening the Didi Chuxing app at the second time point T2 and the second geographical location A2 in the historical records, then the target action is identified as opening the Didi Chuxing app.

[0050] If the first time is 9:00 AM today and the second time is 9:00 AM yesterday, then the time difference between the second time and the first time is 0.

[0051] It is understandable that the user operation corresponding to the second real-time information in the historical record information may be only one or include multiple operations. That is, the target operation determined based on the historical record information may be one or multiple.

[0052] For example, taking the example listed above, if we obtain the second time T2 that matches the first time T1 and the second geographical location A2 that matches the first geographical location A1, and if the user in the historical record only performed the user operation of opening the Didi Chuxing app at the second time T2 and the second geographical location A2, then the determined target operation is opening the Didi Chuxing app; if the user in the historical record performed the user operation of opening both the Didi Chuxing app and the Gaode Map app at the second time T2 and the second geographical location A2, then the determined target operation includes opening both the Didi Chuxing app and the Gaode Map app.

[0053] S103. Query the historical record information to determine the user's frequently used gaze area.

[0054] Based on the user's gaze location information in historical records, a corresponding gaze region can be generated. For example, based on... Figure 3 The gaze point location information shown in the figure generates a gaze point region as follows: Figure 4 As shown.

[0055] In some embodiments, such as Figure 5 As shown, step S103 includes sub-steps S1031 and S1032.

[0056] S1031. Query the historical record information and divide the gaze point location information into a first type of location information and a second type of location information. The first type of location information corresponds to the user not performing a user operation, and the second type of location information corresponds to the user performing a user operation.

[0057] S1032. Determine the frequently operating area of ​​the gaze point based on the second type of location information.

[0058] In practical applications, a user may or may not perform a user action when their gaze rests on a particular point. Therefore, based on whether the user performed a user action at that point, the gaze location information can be divided into two categories: Category 1 location information and Category 2 location information. Category 1 location information indicates that the user did not perform a user action at the corresponding gaze rest point, while Category 2 location information indicates that the user performed a user action at the corresponding gaze rest point. For example, ... Figure 6 As shown in the figure, the red dots correspond to the first type of location information, and the blue dots correspond to the second type of location information.

[0059] Based on the determined second type of location information, the frequently operating region of the fixation point can be determined. For example, based on... Figure 6 The second type of location information shown obtains the gaze point's constant operating region, as follows: Figure 7As shown, since the frequently used area of ​​this gaze point is determined based on the second type of location information and is related to the user's actions, it aligns with the user's behavioral habits, thus further improving the user experience.

[0060] S104. Display the UI corresponding to the target operation in the frequently used operation area of ​​the gaze point.

[0061] For example, the UI corresponding to the target operation includes, but is not limited to, the desktop, application icon, shortcut, etc. For instance, assuming the target operation is determined to be opening the Didi Chuxing app, the Didi Chuxing app icon is displayed in the gaze point's frequently used operation area, or a shortcut to the Didi Chuxing app is displayed in the gaze point's frequently used operation area.

[0062] like Figure 8 As shown, the process for controlling the UI display of AR glasses is as follows:

[0063] The first step is for the user to enable the UI personalization display function;

[0064] The second step is to use an eye tracker to capture the user's eye gaze position each time the user uses AR glasses.

[0065] The third step is to record the user's actions when the user performs them.

[0066] The fourth step is to synchronously record information such as the time of the user's operation, geographical location, and the cumulative number of times the user's operation was performed;

[0067] The fifth step is to generate historical record information based on the records;

[0068] The sixth step is to record the time and location when the user uses the AR glasses again, and then match them with the historical information.

[0069] Step 7: Determine if the match is successful. If the match is successful, proceed to step 8; otherwise, if the match fails, proceed to step 10.

[0070] The eighth step is to predict the target operation that the user will perform and to determine the user's gaze point and frequently used operation area.

[0071] Step 9: Display the UI corresponding to the target operation in the frequently used operation area of ​​the gaze point;

[0072] Step 10: Do not respond.

[0073] In some embodiments, displaying the UI corresponding to the target operation in the gaze point common operation area includes: if the target operation includes multiple operations, displaying multiple application icons corresponding to the target operations in the gaze point common operation area.

[0074] Taking the UI including the application icon corresponding to the target operation as an example, in actual applications, the identified target operation may be one or multiple. For a single target operation, the application icon corresponding to that single target operation is directly displayed in the gaze point's frequent operation area. For multiple target operations, the application icons corresponding to each of the multiple target operations are displayed separately in the gaze point's frequent operation area.

[0075] In some embodiments, displaying application icons corresponding to multiple target operations in the gaze point common operation area includes: determining a weight score for each target operation being performed; and displaying multiple application icons in the gaze point common operation area with different icon sizes, wherein the target operation with a larger weight score has a larger icon size corresponding to it.

[0076] For example, the weight corresponding to the time is pre-set as a1, the weight corresponding to the geographical location as a2, and the weight corresponding to the user operation as a3, where a1 + a2 + a3 = 1. For instance, a1 can be set to 35%, a2 to 35%, and a3 to 30%. It should be noted that the specific values ​​of a1, a2, and a3 can be flexibly set according to the actual situation, and this application does not impose specific restrictions. Based on the second time, second geographical location, cumulative number of user operation executions in the historical record information, and the corresponding weights a1, a2, and a3, the weight score corresponding to each target operation is determined.

[0077] For example, if a1 is 35%, a2 is 35%, a3 is 30%, and the target operations include opening the Didi Chuxing app and opening the Gaode Maps app, and the historical records match the first real-time information with a cumulative total of 24 times opening the Didi Chuxing app and 6 times opening the Gaode Maps app, then the weight score corresponding to opening the Didi Chuxing app can be calculated as 35% + 35% + (24 / 24 + 6).

[0078] *30% = 94%, therefore the weighted score for opening the Gaode Map software is 35% + 35% + (6 / 24 + 6).

[0079] *30% = 76%.

[0080] Next, based on the weight scores corresponding to each target operation, the application icons corresponding to each target operation are displayed in the gaze point's frequently used operation area with different icon sizes. The higher the weight score of the target operation, the larger the corresponding application icon size. For example, using the example listed above, the calculated weight score for opening the Didi Chuxing app is 94%, while the weight score for opening the Gaode Maps app is 76%. That is, the weight score for opening the Didi Chuxing app is greater than the weight score for opening the Gaode Maps app. Therefore, in the gaze point's frequently used operation area, the Didi Chuxing app icon is displayed at a larger size than the Gaode Maps app icon.

[0081] For example, suppose the target operations include a first user operation, a second user operation, a third user operation, and a fourth user operation, and their corresponding application icons are icon 1, icon 2, icon 3, and icon 4, respectively. As described above, the weight score for the first user operation is determined to be q1, the weight score for the second user operation is q2, the weight score for the third user operation is q3, and the weight score for the fourth user operation is q4, where q1 > q2 > q3 > q4. Therefore, icon 1 has the largest icon size, icon 2 has the second largest icon size, icon 3 has a smaller icon size than icon 2, and icon 4 has the smallest icon size. For example, ... Figure 9 As shown, icons 1, 2, 3, and 4 are displayed in different sizes in the frequently used area of ​​the gaze point.

[0082] The weight score reflects the probability that a user will perform a target operation. The higher the weight score of a target operation, the greater the probability that the user will perform that target operation. In other words, the application icon corresponding to the target operation with a higher probability of being performed by the user will be displayed larger, which further improves the user's visual and operational experience.

[0083] It should be noted that, in addition to displaying the application icons corresponding to each target operation at different icon sizes in the gaze point's common operation area, other methods can also be used to display the application icons corresponding to each target operation in the gaze point's common operation area. For example, based on the weight score corresponding to each target operation, the application icons corresponding to each target operation can be displayed at different spatial distances in the gaze point's common operation area, where the target operation with the higher the weight score has, the closer the spatial distance of the corresponding application icon is.

[0084] In the above embodiments, by generating historical information about the user's use of the head-mounted display device, when the user is currently using the head-mounted display device, the system obtains the first real-time information of the user's current use of the head-mounted display device, then queries the historical information to determine the target operation matching the first real-time information, and determines the user's gaze point frequently used operation area. Then, the UI corresponding to the target operation is displayed in the determined gaze point frequently used operation area. This allows the UI layout of the head-mounted display device to be set according to the user's behavior, rather than being fixed and singular, thus enhancing the flexibility of the UI display. In this way, placing the UI corresponding to the target operation of the current local user intent in the gaze point frequently used operation area also facilitates the user to perform the target operation, thereby improving the user experience.

[0085] Please see Figure 10 , Figure 10 This is a schematic block diagram of a UI display device provided in an embodiment of this application. The UI display device can be configured in a head-mounted display device to perform the aforementioned UI user interface display method.

[0086] like Figure 10 As shown, the UI display device 200 may include a processor 210 and a memory 220, wherein the processor 210 and the memory 220 are connected via a bus, such as an I2C (Inter-integrated Circuit) bus.

[0087] Specifically, the processor 210 can be a microcontroller unit (MCU), a central processing unit (CPU), or a digital signal processor (DSP), etc.

[0088] Specifically, the memory 220 can be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a portable hard drive, etc. The memory 220 stores various computer programs for the processor 210 to execute.

[0089] The processor 210 is configured to run a computer program stored in the memory, and to perform the following steps when executing the computer program:

[0090] Obtain first-hand real-time information about the user's current head-mounted display device;

[0091] Query the user's historical usage history of the head-mounted display device to determine the target operation that matches the first real-time information;

[0092] Query the historical record information to determine the user's frequently used gaze area;

[0093] The UI corresponding to the target operation is displayed in the frequently used area of ​​the gaze point.

[0094] In some embodiments, before acquiring the first real-time information about the user's current use of the head-mounted display device, the processor 210 is configured to perform the following:

[0095] Each time a user uses the head-mounted display device, the system records the user's gaze point location information and the relevant element information corresponding to the user's operation.

[0096] The historical record information is generated based on the relevant element information and the gaze point location information.

[0097] In some embodiments, the relevant element information includes user operations and second real-time information corresponding to the execution of the user operations. When the processor 210 implements the querying of the user's historical usage history information of the head-mounted display device and determines the target operation matching the first real-time information, it is configured to:

[0098] Query the historical record information to see if there is a second real-time information that matches the first real-time information;

[0099] If so, the user operation corresponding to the matched second real-time information is determined as the target operation.

[0100] In some embodiments, the first real-time information includes at least one of a first time and a first geographical location, and the second real-time information includes at least one of a second time and a second geographical location. When the processor 210 performs the query to determine whether there is second real-time information in the historical record information that matches the first real-time information, it is configured to:

[0101] If the historical record information shows that the time difference between the second time and the first time is less than a preset time threshold, and / or if the historical record information shows that the distance between the second geographical location and the first geographical location is less than a preset distance threshold, then it is determined that the historical record information contains second real-time information that matches the first real-time information.

[0102] In some embodiments, when the processor 210 performs the querying of the historical record information and determines the user's frequently used gaze area, it is configured to:

[0103] The historical records are queried, and the gaze point location information is divided into a first type of location information and a second type of location information. The first type of location information corresponds to the user not performing a user operation, and the second type of location information corresponds to the user performing a user operation.

[0104] Based on the second type of location information, the frequently operating area of ​​the gaze point is determined.

[0105] In some embodiments, the UI includes an application icon corresponding to the target operation, and the processor 210, when implementing the display of the UI corresponding to the target operation in the gaze point's frequently used operation area, is configured to:

[0106] If the target operation includes multiple operations, then multiple application icons corresponding to the target operations are displayed in the gaze point common operation area.

[0107] In some embodiments, when the processor 210 implements the display of application icons corresponding to multiple target operations in the frequently used area of ​​the gaze point, it is configured to:

[0108] Determine the weight score for each of the target operations to be performed;

[0109] Multiple application icons are displayed in the gaze point's frequently used operation area with different icon sizes, wherein the target operation with a higher weight score corresponds to a larger icon size for the application icon.

[0110] The UI display device 200 can execute the UI display method provided in the embodiments of this application. Therefore, it can achieve the beneficial effects that the UI display method provided in the embodiments of this application can achieve. For details, please refer to the previous embodiments, which will not be repeated here.

[0111] This application embodiment also provides a head-mounted display device, which includes, but is not limited to, AR glasses, VR glasses, etc., wherein the head-mounted display device includes a UI display device, which can be... Figure 10 The UI display device 200 shown is illustrated. Therefore, the head-mounted display device can achieve the beneficial effects of the UI display method provided in the embodiments of this application, as detailed in the preceding embodiments, which will not be repeated here.

[0112] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the UI display method described above.

[0113] The computer-readable storage medium may be an internal storage unit of the UI display device or head-mounted display device described in the foregoing embodiments, such as the hard disk or memory of the UI display device or head-mounted display device. The computer-readable storage medium may also be an external storage device of the UI display device or head-mounted display device, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD card), flash card, etc., provided on the UI display device or head-mounted display device.

[0114] Since the computer program stored in the storage medium can execute any of the UI display methods provided in the embodiments of this application, it can achieve the beneficial effects that any of the UI display methods provided in the embodiments of this application can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0115] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0116] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the scope of the technology disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application.

Claims

1. A method for displaying a UI (User Interface), characterized in that, The UI display method includes: Obtain first real-time information about the user's current use of the head-mounted display device; the first real-time information includes at least one of a first moment and a first geographical location; Query the user's historical usage history of the head-mounted display device to determine the target operation that matches the first real-time information; Query the historical record information to determine the user's frequently used gaze area; The UI corresponding to the target operation is displayed in the frequently used area of ​​the gaze point; The step of querying the user's historical usage history of the head-mounted display device to determine the target operation matching the first real-time information includes: The system queries the historical records to determine if there is any second real-time information that matches the first real-time information; the second real-time information includes at least one of a second time and a second geographical location. If so, the user operation corresponding to the matched second real-time information is determined as the target operation; The step of querying whether there is second real-time information in the historical record information that matches the first real-time information includes: If the historical record information shows that the time difference between the second time and the first time is less than a preset time threshold, and / or if the historical record information shows that the distance between the second geographical location and the first geographical location is less than a preset distance threshold, then it is determined that the historical record information contains second real-time information that matches the first real-time information.

2. The UI display method as described in claim 1, characterized in that, Before obtaining the first real-time information about the user's current use of the head-mounted display device, the process includes: Each time a user uses the head-mounted display device, the system records the user's gaze point location information and the relevant element information corresponding to the user's operation. The historical record information is generated based on the relevant element information and the gaze point location information.

3. The UI display method as described in claim 2, characterized in that, The relevant information includes user operations and the second real-time information corresponding to the execution of the user operations.

4. The UI display method as described in claim 3, characterized in that, The process of querying the historical data to determine the user's frequently used gaze area includes: The historical records are queried, and the gaze point location information is divided into a first type of location information and a second type of location information. The first type of location information corresponds to the user not performing a user operation, and the second type of location information corresponds to the user performing a user operation. Based on the second type of location information, the frequently operating area of ​​the gaze point is determined.

5. The UI display method according to any one of claims 1 to 4, characterized in that, The UI includes an application icon corresponding to the target operation, and displaying the UI corresponding to the target operation in the frequently used area of ​​the gaze point includes: If the target operation includes multiple operations, then multiple application icons corresponding to the target operations are displayed in the gaze point common operation area.

6. The UI display method as described in claim 5, characterized in that, The display of multiple application icons corresponding to the target operation in the frequently used area of ​​the gaze point includes: Determine the weight score for each of the target operations to be performed; Multiple application icons are displayed in the gaze point's frequently used operation area with different icon sizes, wherein the target operation with a higher weight score corresponds to a larger icon size for the application icon.

7. A UI display device, characterized in that, The UI display device includes a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and, when executing the computer program, implement the steps of the UI display method as described in any one of claims 1 to 6.

8. A head-mounted display device, characterized in that, The head-mounted display device includes the UI display device as described in claim 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the UI display method as described in any one of claims 1 to 6.

Images