Method, device, medium and equipment for controlling proportion of AR / VR / MR glasses function interface

Abstract

The present disclosure relates to a method, device, medium and equipment for controlling the proportion of AR / VR / MR glasses function interface, which comprises the following steps: determining the visual angle range information of the corresponding eye features of a user, generating a preset initial proportion control interface in the corresponding display area according to the visual angle range information, determining the display order of the function icons corresponding to each set function in the line-of-sight range corresponding to the visual angle range information from the initial proportion control interface, determining the first visual attention function icon corresponding to the visual attention point information according to the display order and the function icon, the visual attention point information and the target function icon, determining the second visual attention function icon based on the visual attention point information, generating a display proportion locking instruction to lock the display content and display proportion of the corresponding visual attention function icon in the current preset initial proportion control interface, and generating a target proportion control interface. Thus, the operation efficiency of the multifunctional operation interface is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of smart glasses, in particular, to an AR / VR / MR glasses function interface proportion control method, device, medium and equipment. BACKGROUND

[0002] The working principle of AR / VR / MR glasses is that the projection device in the glasses projects the image to be presented onto the lens on the side of the glasses facing the wearer's eyes, and uses the reflection function of the lens to reflect the above image light to the wearer's eyes, thereby forming an overlap of the real scene and the virtual image on the wearer's retina, realizing the perception of augmented reality. Since the lens of the AR / VR / MR glasses is transparently arranged, when the proportion between the light rays (incident light rays) incident on the lens from the front of the wearer and the light rays (reflected light rays) reflected by the virtual image through the lens is not coordinated, the virtual image presented in the wearer's eyes will become too dark or too bright relative to the real scene, affecting the wearer's perception. In addition, when wearing AR glasses to perform mechanical or other operations, it is also necessary to flexibly adjust the brightness relationship between the real scene and the virtual image on the lens according to the actual working conditions. SUMMARY

[0003] In order to overcome the problems in the related art, the present disclosure provides an AR / VR / MR glasses function interface proportion control method.

[0004] According to a first aspect of an embodiment of the present disclosure, an AR / VR / MR glasses function interface proportion control method is provided, applied to a wearable device, the wearable device including an augmented reality (AR) glasses, a virtual reality (VR) glasses or a mixed reality (MR) glasses, and the method includes:

[0005] In response to identifying an eye feature of a user based on the wearable device, determining the visual angle range information of the corresponding eye feature of the user, and generating a preset initial proportion control interface in the corresponding display area according to the visual angle range information;

[0006] According to the function display sequence of the wearable device, determining the display sequence of each set function corresponding to the function icon in the visual line range corresponding to the visual angle range information from the initial proportion control interface, and displaying the target function icon in the preset initial proportion control interface one by one according to the display sequence and the function icon;

[0007] based on the user eye feature, determine visual attention point information of the user within a visual angle range, and based on the visual attention point information and the target function icon, determine a first visual attention function icon corresponding to the visual attention point information, and display the first visual attention function icon in an enlarged manner in the preset initial proportion control interface;

[0008] monitor the position of the visual attention point information in the preset initial proportion control interface in real time, and in response to the point corresponding to the visual attention point information leaving a visual area corresponding to the first visual attention function icon in the preset initial proportion control interface, display the first visual attention function icon in a reduced manner in the preset initial proportion control interface, and based on the position of the visual attention point information in the preset initial proportion control interface, determine a second visual attention function icon, and display the second visual attention function icon in an enlarged manner;

[0009] In a case where it is determined that the visual residence time of the user eye feature in the second visual attention function icon is greater than a set time threshold, a display proportion locking instruction is generated to lock the display content and display proportion of the corresponding visual attention function icon in the preset initial proportion control interface, and a target proportion control interface is generated.

[0010] Optionally, the method further comprises:

[0011] obtaining display position information of each visual attention function icon under the target proportion control interface and target position information of the second visual attention function icon;

[0012] In a case where it is determined that the visual position corresponding to the user eye feature stays in a position of another visual attention function icon outside the target position information for a time greater than a set time threshold, a display proportion unlocking instruction is generated, which is used to restore the display proportion of the visual attention function icon in the target proportion control interface to a set display proportion, and according to the target function icon in the preset initial proportion control interface, display subsequent function icons one by one.

[0013] Optionally, the determining, based on the visual attention point information and the target function icon, of the first visual attention function icon corresponding to the visual attention point information comprises:

[0014] determining the residence time of the user corresponding visual angle on each target function icon according to the position of the target function icon in the initial proportion control interface and the visual attention point corresponding to the user eye feature;

[0015] If the residence time is greater than a set time threshold, a pupil attention amplification image corresponding to the user is determined according to the user eye features, and in a case where the pupil attention amplification image matches a preset pupil focus image, the first visual attention function icon corresponding to the visual attention point information under the preset initial proportion control interface is obtained.

[0016] Optionally, the method further comprises:

[0017] The pupil proportion feature corresponding to the pupil attention amplification image is generated based on a preset feature convolution network extracting a pupil proportion feature of the pupil attention amplification image.

[0018] A set pupil proportion feature corresponding to the preset pupil focus image is obtained.

[0019] In a case where the pupil proportion feature matches the set pupil proportion feature, a proportion amplification locking instruction is generated, and the proportion amplification locking instruction is used to amplify the first visual attention function icon and lock the preset initial proportion control interface currently displayed.

[0020] In a case where the pupil proportion feature does not match the set pupil proportion feature, the target function icons of the preset initial proportion control interface are displayed one by one.

[0021] Optionally, the method further comprises:

[0022] In response to displaying a target proportion control interface in a display area corresponding to the visual angle range information, a plurality of function icons selected by the user in the target proportion control interface are determined according to the user eye features.

[0023] A target proportion interface is generated according to the plurality of function icons and non-attention function icons, the target proportion interface is used to display the plurality of function icons based on a preset display proportion and hide the non-attention function icons, and the non-attention function icons are one or more function icons that are not selected.

[0024] Optionally, the target proportion interface is generated according to the plurality of function icons and non-attention function icons, and the target proportion interface comprises:

[0025] The display proportion and display area of the plurality of function icons in the target proportion interface are determined.

[0026] A target scene matching the plurality of function icons is obtained from the preset scene library.

[0027] The target proportion interface is generated based on the display proportion and display area in the target scene according to the target scene and the plurality of function icons.

[0028] According to a second aspect of the embodiments of the present disclosure, a device for controlling an AR / VR / MR glasses function interface proportion is provided, which is applied to a wearable device including an augmented reality (AR) glasses, a virtual reality (VR) glasses or a mixed reality (MR) glasses. The method includes:

[0029] A generating module is configured to, in response to identifying an eye feature of a user based on the wearable device, determine a visual angle range information of the eye feature of the user, and generate a preset initial proportion control interface in a corresponding display area according to the visual angle range information.

[0030] A display module is configured to, according to a function display sequence of the wearable device, determine a display sequence of a function icon corresponding to each setting function in a visual line range corresponding to the visual angle range information from the initial proportion control interface, and display the target function icon in the preset initial proportion control interface one by one according to the display sequence and the function icon.

[0031] A first determining module is configured to identify an eye feature of a user during a display process based on the wearable device, determine visual attention point information of the user in a visual angle range according to the eye feature of the user, determine a first visual attention function icon corresponding to the visual attention point information according to the visual attention point information and the target function icon, and display the first visual attention function icon in the preset initial proportion control interface.

[0032] A second determining module is configured to monitor a position of the visual attention point information in the preset initial proportion control interface in real time, in response to a point corresponding to the visual attention point information leaving a visual area corresponding to the first visual attention function icon in the preset initial proportion control interface, display the first visual attention function icon in the preset initial proportion control interface in a reduced manner, and determine a second visual attention function icon based on the position of the visual attention point information in the preset initial proportion control interface, and display the second visual attention function icon in an enlarged manner.

[0033] An executing module is configured to, in a case where a visual residence time of the eye feature of the user in the second visual attention function icon is greater than a set time threshold, generate a display proportion locking instruction to lock display content and a display proportion of a corresponding visual attention function icon in the preset initial proportion control interface, and generate a target proportion control interface.

[0034] Optionally, the device further includes a determining module configured to:

[0035] Obtain display position information of each visual attention function icon under the target proportion control interface and target position information of a second visual attention function icon.

[0036] In a case where the user's eye feature corresponds to a visual position outside the target position information, and the staying time of the visual position in the corresponding position of the other visual attention function icon is greater than a set time threshold, a display proportion unlocking instruction is generated, the display proportion unlocking instruction is used to restore the display proportion of the visual attention function icon in the target proportion control interface to a set display proportion, and according to the target function icon in the preset initial proportion control interface, the subsequent function icons are displayed one by one.

[0037] According to a third aspect of embodiments of the present disclosure, a computer readable storage medium is provided, and the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the method in any one of the first aspect of the present disclosure.

[0038] According to a fourth aspect of embodiments of the present disclosure, an electronic device is provided, and the electronic device comprises:

[0039] a processor;

[0040] a memory for storing processor executable instructions;

[0041] The processor is configured to execute the executable instructions in the memory to implement the method in any one of the first aspect of the present disclosure.

[0042] The technical solutions provided by the embodiments of the present disclosure can include the following beneficial effects:

[0043] In response to identifying the eye feature of the user based on the wearable device, the view angle range information of the corresponding eye feature of the user is determined, a preset initial proportion control interface is generated in the corresponding display area according to the view angle range information, the display order of the function icons corresponding to each setting function in the line-of-sight range corresponding to the view angle range information is determined from the initial proportion control interface according to the functional display order of the wearable device, and the target function icon in the preset initial proportion control interface is displayed one by one according to the display order and the function icon. Based on the identification of the eye feature of the user during the display process by the wearable device, the visual attention point information of the user in the view angle range is determined according to the eye feature of the user, the first visual attention function icon corresponding to the visual attention point information is determined according to the visual attention point information and the target function icon, and the first visual attention function icon is displayed in the preset initial proportion control interface. The position of the visual attention point information in the preset initial proportion control interface is monitored in real time, the first visual attention function icon is displayed in the preset initial proportion control interface in response to the position corresponding to the visual attention point information leaving the visual area corresponding to the first visual attention function icon, the second visual attention function icon is determined based on the position of the visual attention point information in the preset initial proportion control interface, and the second visual attention function icon is displayed. In the case where the visual residence time of the eye feature of the user in the second visual attention function icon is greater than a set time threshold, a display proportion locking instruction is generated to lock the display content and display proportion of the corresponding visual attention function icon in the preset initial proportion control interface, and a target proportion control interface is generated. Thus, the display proportion of the wearable device can be adjusted based on the visual attention point to adapt to the visual control requirements of the wearable device, and the operation efficiency of the multifunctional operation interface is improved.

[0044] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and are not limiting of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0045] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure and serve to explain the principles of the present disclosure.

[0046] Figure 1 is a flowchart of an AR / VR / MR glasses function interface proportion control method according to an exemplary embodiment.

[0047] Figure 2is a block diagram of a control device for AR / VR / MR glasses function interface proportion according to an example embodiment. DETAILED DESCRIPTION

[0048] The example embodiments will be described in detail herein with reference to the attached drawings. In the following description, unless otherwise indicated, like numbers refer to like elements throughout the description and drawings. The following description is not meant to limit the application to all of the embodiments described herein. Rather, the following description is meant to provide an example of devices and methods consistent with the application as detailed in the appended claims.

[0049] It should be noted that all the actions of obtaining signals, information or data in this application are carried out in accordance with the corresponding data protection regulations and policies of the country where the device is located, and with the authorization of the corresponding device owner.

[0050] Figure 1 is a flowchart of a control method for AR / VR / MR glasses function interface proportion according to an example embodiment. Referring to Figure 1 As shown, the method comprises:

[0051] In step S11, in response to the display device starting the anti-peeping driving mode, it is determined whether there is an unauthorized user in the set visual angle range of the display device through the front camera, and if there is the unauthorized user, the head image features of the unauthorized user are obtained based on the front camera.

[0052] For example, the embodiment is applied to a wearable device, which can be an AR glasses, VR glasses or MR glasses, etc. The wearable device is used to realize virtual reality function. In the embodiment, the display interface proportion of the smart glasses in the display process is controlled. When the eye features of the user are recognized based on the wearable device, i.e. when the wearable device is used by the user, the eye features of the corresponding eyes of the current user are determined, the visual angle range information is determined, and a preset initial proportion control interface is generated in the display area of the corresponding smart glasses according to the visual angle range.

[0053] In step S12, according to the function display sequence of the wearable device, the display sequence of each set function corresponding to the function icon in the visual line range corresponding to the visual angle range information is determined from the initial proportion control interface, and the target function icon in the preset initial proportion control interface is displayed one by one according to the display sequence and the function icon.

[0054] In an example, the function area display in the proportion control interface in the embodiment is displayed one by one according to the function display order, so as to avoid too many function icons and too large display area, and increase the operability of function display. In the embodiment, the corresponding function display order can be set based on the number of clicks of the corresponding icon. From the initial proportion control interface, the display order of the function icons corresponding to each setting function in the visual line range corresponding to the visual angle range information is determined, and the target function icon in the preset initial proportion control interface is displayed one by one according to the display order and the function icon.

[0055] In step S13, the user's eye features in the display process are identified based on the wearable device, the visual attention point position information of the user in the visual angle range is determined according to the user's eye features, the first visual attention function icon corresponding to the visual attention point position information is determined according to the visual attention point position information and the target function icon, and the first visual attention function icon is displayed in the preset initial proportion control interface.

[0056] In an example, the eye features of the user in the display process are identified based on the wearable device, the visual attention point position information of the user in the visual angle range is determined according to the user's eye features, the first visual attention function icon corresponding to the visual attention point position information is determined according to the visual attention point position information and the target function icon, and the first visual attention function icon is displayed in the preset initial proportion control interface. Through visual tracking, different proportion display of each display icon in the smart glasses is realized, so as to realize intelligent display control of the wearable device, and to ensure the control performance of the wearable device.

[0057] In step S14, the position of the visual attention point position information in the preset initial proportion control interface is monitored in real time, and in response to the point corresponding to the visual attention point position information leaving the visual area corresponding to the first visual attention function icon in the preset initial proportion control interface, the first visual attention function icon is displayed in the preset initial proportion control interface. The second visual attention function icon is determined based on the position of the visual attention point position information in the preset initial proportion control interface, and the second visual attention function icon is displayed.

[0058] In an embodiment, the visual attention point of the user during use of the wearable device is identified, and after the visual attention function icon is displayed in a magnified manner, the position of the visual attention point in the preset initial proportion control interface is monitored in real time. When the visual attention point exits the visual area corresponding to the visual attention function icon in the preset initial proportion control interface, the visual attention function icon is displayed in a reduced manner in the preset initial proportion control interface, and a second visual attention function icon is determined based on the position of the visual attention point in the preset initial proportion control interface, and the visual attention function icon is displayed in a magnified manner.

[0059] In step S15, in a case where it is determined that the visual residence time of the user's eye feature in the second visual attention function icon is greater than the set time threshold, a display proportion locking instruction is generated to lock the display content and display proportion of the corresponding visual attention function icon in the current preset initial proportion control interface, and a target proportion control interface is generated.

[0060] In an embodiment, the visual attention point of the user during use of the wearable device is identified, and after the visual attention function icon is displayed in a magnified manner, the position of the visual attention point in the preset initial proportion control interface is monitored in real time. When the visual attention point exits the visual area corresponding to the visual attention function icon in the preset initial proportion control interface, the visual attention function icon is displayed in a reduced manner in the preset initial proportion control interface, and a second visual attention function icon is determined based on the position of the visual attention point in the preset initial proportion control interface, and the visual attention function icon is displayed in a magnified manner.

[0061] In response to identifying the eye feature of the user based on the wearable device, the range of view angles of the corresponding eye feature of the user is determined, and a preset initial proportion control interface is generated in the corresponding display area according to the range of view angles. According to the functional display sequence of the wearable device, the display sequence of the function icons corresponding to each setting function in the line-of-sight range corresponding to the range of view angles is determined from the initial proportion control interface, and the target function icon in the preset initial proportion control interface is displayed one by one according to the display sequence and the function icon. Based on the identification of the eye feature of the user during the display process by the wearable device, the visual attention point information of the user in the range of view angles is determined according to the eye feature of the user, the first visual attention function icon corresponding to the visual attention point information is determined according to the visual attention point information and the target function icon, and the first visual attention function icon is displayed in the preset initial proportion control interface. The position of the visual attention point information in the preset initial proportion control interface is monitored in real time, and in response to the point corresponding to the visual attention point information leaving the visual area corresponding to the first visual attention function icon in the preset initial proportion control interface, the first visual attention function icon is displayed in the preset initial proportion control interface. The display is reduced, and the second visual attention function icon is determined based on the position of the visual attention point information in the preset initial proportion control interface, and the second visual attention function icon is displayed. In the case where the visual residence time of the eye feature of the user in the second visual attention function icon is greater than a set time threshold, a display proportion locking instruction is generated to lock the display content and display proportion of the corresponding visual attention function icon in the preset initial proportion control interface, and a target proportion control interface is generated. Therefore, the display proportion of the wearable device can be adjusted based on the visual attention point to adapt to the visual control requirements of the wearable device, and the operation efficiency of the multifunctional operation interface is improved.

[0062] Optionally, in some embodiments, the method further comprises:

[0063] Obtaining display position information of each visual attention function icon under the target proportion control interface and target position information of the second visual attention function icon;

[0064] In the case where the residence time of the visual position corresponding to the eye feature of the user in the corresponding position of the other visual attention function icon outside the target position information is greater than a set time threshold, a display proportion unlocking instruction is generated, and the display proportion unlocking instruction is used to restore the display proportion of the visual attention function icon in the target proportion control interface to a set display proportion, and the subsequent function icons are displayed one by one according to the target function icons in the preset initial proportion control interface.

[0065] In some embodiments, after the target proportion control interface is generated and displayed on the wearable device, the display position information of each visual attention function icon and the target position information of the second visual attention function icon are obtained. When it is determined that the user's eyes stay at the position of the other visual attention function icon outside the target position information for more than a set time threshold, a display proportion unlocking instruction is generated. The display proportion unlocking instruction is used to restore the display proportion of the visual attention function icon in the target proportion control interface to a set display proportion, and the subsequent function icons are displayed one by one according to the target function icon in the preset initial proportion control interface.

[0066] Optionally, in some embodiments, the step S13 comprises:

[0067] According to the position of the target function icon in the initial proportion control interface and the visual attention point corresponding to the user's eye features, the residence time of the user's corresponding visual angle on each target function icon is determined.

[0068] If the residence time is greater than a set time threshold, the user's corresponding pupil attention amplification image is determined according to the user's eye features. In the case where the pupil attention amplification image matches the preset pupil focus image, the first visual attention function icon corresponding to the visual attention point information under the current preset initial proportion control interface is obtained.

[0069] In some embodiments, according to the position of the target function icon in the initial proportion control interface and the visual attention point corresponding to the user's eye features, the residence time of the user's corresponding visual angle on each target function icon is determined. If the residence time is greater than a set time threshold, the user's corresponding pupil attention amplification image is determined according to the user's eye features. In the case where the pupil attention amplification image matches the preset pupil focus image, the first visual attention function icon corresponding to the visual attention point information under the current preset initial proportion control interface is obtained.

[0070] Optionally, in some embodiments, the method further comprises:

[0071] Based on the preset feature convolution network, the pupil proportion feature of the pupil attention amplification image is extracted, and the pupil proportion feature corresponding to the pupil attention amplification image is generated.

[0072] The set pupil proportion feature corresponding to the preset pupil focus image is obtained.

[0073] In the case where the pupil proportion feature matches the set pupil proportion feature, a proportion amplification locking instruction is generated. The proportion amplification locking instruction is used to amplify the first visual attention function icon and lock the current display of the preset initial proportion control interface.

[0074] In the case that the pupil proportion feature does not match the set pupil proportion feature, the target function icons of the preset initial proportion control interface are displayed one by one.

[0075] In an example, in the embodiment, the pupil proportion feature of the pupil attention amplification image is extracted based on the preset feature convolution network, the pupil proportion feature corresponding to the pupil attention amplification image is generated, the set pupil proportion feature corresponding to the preset pupil focus image is obtained, in the case that the pupil proportion feature matches the set pupil proportion feature, the proportion amplification locking instruction is generated, the proportion amplification locking instruction is used to amplify the first visual attention function icon and lock the preset initial proportion control interface currently displayed, and in the case that the pupil proportion feature does not match the set pupil proportion feature, the target function icons of the preset initial proportion control interface are displayed one by one.

[0076] Optionally, in some embodiments, the method further includes:

[0077] In response to displaying the target proportion control interface in the display area corresponding to the visual angle range information, a plurality of function icons selected by the user in the target proportion control interface are determined according to the user eye feature;

[0078] A target proportion interface is generated according to the plurality of function icons and a non-attention function icon, the target proportion interface is used to display the plurality of function icons based on a preset display proportion and hide the non-attention function icon, and the non-attention function icon is one or more function icons that are not selected.

[0079] In an example, in the embodiment, in response to displaying the target proportion control interface in the display area corresponding to the visual angle range information, a plurality of function icons selected by the user in the target proportion control interface are determined according to the user eye feature, and a target proportion interface is generated according to the plurality of function icons and a non-attention function icon, the target proportion interface is used to display the plurality of function icons based on a preset display proportion and hide the non-attention function icon, and the non-attention function icon is one or more function icons that are not selected.

[0080] Optionally, in some embodiments, the method further includes:

[0081] A display proportion and a display area of the plurality of function icons in the target proportion interface are determined.

[0082] A target scene matching the plurality of function icons is obtained from the preset scene library.

[0083] According to the target scene and the plurality of function icons, a target proportion interface is generated in the target scene based on the display proportion and the display area.

[0084] In an example, the wearable device in this embodiment determines the display proportion and display area of the plurality of function icons in the target proportion interface, obtains a target scene matching the plurality of function icons from a preset scene library, and generates the target proportion interface in the target scene based on the display proportion and display area according to the target scene and the plurality of function icons.

[0085] Figure 2 is a block diagram of an AR / VR / MR glasses function interface proportion control device according to an embodiment of the present disclosure. As shown in the figure, the device 100 includes: Figure 2

[0086] The generating module 110 is configured to, in response to identifying the eye features of the user based on the wearable device, determine the visual angle range information of the corresponding eye features of the user, and generate a preset initial proportion control interface in the corresponding display area according to the visual angle range information.

[0087] The display module 120 is configured to determine the display order of each set function icon in the visual line range corresponding to the visual angle range information in the initial proportion control interface according to the function display order of the wearable device, and display the target function icon in the preset initial proportion control interface according to the display order and the function icon.

[0088] The first determining module 130 is configured to identify the eye features of the user during the display process based on the wearable device, determine the visual attention point position information of the user in the visual angle range according to the eye features of the user, determine the first visual attention function icon corresponding to the visual attention point position information according to the visual attention point position information and the target function icon, and display the first visual attention function icon in the preset initial proportion control interface.

[0089] The second determining module 140 is configured to monitor the position of the visual attention point position information in the preset initial proportion control interface in real time, in response to the position corresponding to the visual attention point position information leaving the visual area corresponding to the first visual attention function icon in the preset initial proportion control interface, display the first visual attention function icon in the preset initial proportion control interface in a reduced manner, and determine the second visual attention function icon based on the position of the visual attention point position information in the preset initial proportion control interface, and display the second visual attention function icon in an enlarged manner.

[0090] ​The execution module 150 is configured to generate a display proportion locking instruction to lock the display content and display proportion of the corresponding visual attention function icon in the preset initial proportion control interface, and generate a target proportion control interface, in a case where it is determined that the visual residence time of the eye feature of the user in the second visual attention function icon is greater than a set time threshold.

[0091] In the foregoing manner, in response to identifying the eye feature of the user based on the wearable device, the visual angle range information of the corresponding eye feature of the user is determined, a preset initial proportion control interface is generated in the corresponding display area according to the visual angle range information, the display order of each set function corresponding function icon in the visual line range corresponding to the visual angle range information is determined from the initial proportion control interface according to the function display order of the wearable device, and the target function icon in the preset initial proportion control interface is displayed one by one according to the display order and the function icon. Based on the identification of the eye feature of the user during the display process by the wearable device, the visual attention point position information of the user in the visual angle range is determined according to the eye feature of the user, the first visual attention function icon corresponding to the visual attention point position information is determined according to the visual attention point position information and the target function icon, and the first visual attention function icon is displayed in a magnified manner in the preset initial proportion control interface. The position of the visual attention point position information in the preset initial proportion control interface is monitored in real time, in response to the point position corresponding to the visual attention point position information leaving the visual area corresponding to the first visual attention function icon in the preset initial proportion control interface, the first visual attention function icon is displayed in a reduced manner in the preset initial proportion control interface, and the second visual attention function icon is determined based on the position of the visual attention point position information in the preset initial proportion control interface, and the second visual attention function icon is displayed in a magnified manner. In a case where it is determined that the visual residence time of the eye feature of the user in the second visual attention function icon is greater than a set time threshold, a display proportion locking instruction is generated to lock the display content and display proportion of the corresponding visual attention function icon in the preset initial proportion control interface, and a target proportion control interface is generated. Thus, the display proportion of the wearable device can be adjusted based on the visual attention point position, to adapt to the visual control requirements of the wearable device, and improve the operation efficiency of the multi-function operation interface.

[0092] The embodiment of the present disclosure further provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement the steps of the method in any one of the foregoing embodiments.

[0093] The embodiment of the present disclosure further provides an electronic device, which comprises:

[0094] a processor;

[0095] a memory for storing processor-executable instructions;

[0096] wherein the processor is configured to execute the executable instructions in the memory to implement the method of any of the preceding embodiments.

[0097] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure. It is intended that the specification and examples be considered as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

[0098] It is to be understood that the disclosure is not limited to the precise structures herein described and illustrated in the drawings, and that various modifications and changes can be made without departing from its scope. The scope of the disclosure is limited only by the claims that follow.

Claims

1. A method for controlling a proportion of a function interface of an AR / VR / MR glasses, characterized in that, Applied to a wearable device, the wearable device comprising an augmented reality (AR) glasses, a virtual reality (VR) glasses or a mixed reality (MR) glasses, the method comprises: In response to identifying an eye feature of a user based on the wearable device, determining a visual angle range information of the corresponding eye feature of the user, and generating a preset initial proportion control interface in the corresponding display area according to the visual angle range information; According to the functional display sequence of the wearable device, determining the display sequence of each set function corresponding to the functional icon in the visual angle range information corresponding to the line of sight range from the initial proportion control interface, and displaying the target functional icon in the preset initial proportion control interface according to the display sequence and the functional icon; Based on the identification of the user's eye feature during the display process by the wearable device, determining the visual attention point information of the user in the visual angle range according to the user's eye feature, and determining the first visual attention functional icon corresponding to the visual attention point information according to the visual attention point information and the target functional icon, and displaying the first visual attention functional icon in the preset initial proportion control interface; According to the visual attention point information and the target functional icon, the first visual attention functional icon corresponding to the visual attention point information is determined, which comprises: According to the position of the target functional icon in the initial proportion control interface and the visual attention point corresponding to the user's eye feature, the residence time of the user's corresponding visual angle on each target functional icon is determined; If the residence time is greater than a set time threshold, the user's corresponding pupil attention magnification image is determined according to the user's eye feature, and the first visual attention functional icon corresponding to the visual attention point information under the current preset initial proportion control interface is obtained in the case that the pupil attention magnification image matches the preset pupil focus image; Real-time monitoring the position of the visual attention point information in the preset initial proportion control interface, in response to the point corresponding to the visual attention point information leaving the visual area corresponding to the first visual attention functional icon in the preset initial proportion control interface, the first visual attention functional icon is displayed in the preset initial proportion control interface, and the second visual attention functional icon is determined based on the position of the visual attention point information in the preset initial proportion control interface, and the second visual attention functional icon is displayed; In the case that the visual residence time of the user's eye feature in the second visual attention functional icon is greater than a set time threshold, a display proportion locking instruction is generated to lock the display content and display proportion of the corresponding visual attention functional icon in the current preset initial proportion control interface, and a target proportion control interface is generated; The method further comprises: Based on the preset feature convolution network, the pupil proportion feature of the pupil attention magnification image is extracted, and the pupil proportion feature corresponding to the pupil attention magnification image is generated; The set pupil proportion feature corresponding to the preset pupil focus image is obtained; In a case where the pupil proportion feature matches the set pupil proportion feature, a proportion zoom-in locking instruction is generated, the proportion zoom-in locking instruction being used to zoom in the first visual attention function icon and lock a preset initial proportion control interface currently displayed; In a case where the pupil proportion feature does not match the set pupil proportion feature, the target function icon of the preset initial proportion control interface is displayed one by one.

2. The control method of AR / VR / MR glasses function interface proportion according to claim 1, characterized in that, The method further comprises: obtaining display position information of each visual attention function icon under the target proportion control interface and target position information of a second visual attention function icon; In a case where a stay time of the user's eye feature corresponding visual position at a corresponding position of other visual attention function icons outside the target position information is greater than a set time threshold, a display proportion unlocking instruction is generated, the display proportion unlocking instruction being used to restore a display proportion of visual attention function icons in the target proportion control interface to a set display proportion, and the one-by-one display of subsequent function icons is performed according to the target function icon in the preset initial proportion control interface.

3. The AR / VR / MR glasses function interface proportion control method according to claim 1, characterized in that, The method further comprises: in response to displaying the target proportion control interface in the display area corresponding to the visual angle range information, determining a plurality of function icons selected by the user in the target proportion control interface according to the user's eye feature; generating a target proportion interface according to the plurality of function icons and non-attention function icons, the target proportion interface being used to display the plurality of function icons based on a preset display proportion and hide the non-attention function icons, which are one or more function icons not selected.

4. The AR / VR / MR glasses function interface proportion control method according to claim 3, characterized in that, The generation of the target proportion interface according to the plurality of function icons and non-attention function icons comprises: determining a display proportion and a display area of the plurality of function icons in the target proportion interface; obtaining a target scene matched with the plurality of function icons from a preset scene library; generating the target proportion interface based on the display proportion and the display area in the target scene according to the target scene and the plurality of function icons.

5. An AR / VR / MR glasses function interface proportion control device, characterized in that, Applied to a wearable device, the wearable device comprising an augmented reality (AR) glasses, a virtual reality glasses or a mixed reality glasses, the apparatus comprises: a generation module configured to, in response to identifying an eye feature of a user based on the wearable device, determine visual angle range information of the eye feature corresponding to the user, and generate a preset initial proportion control interface in a corresponding display area according to the visual angle range information; a display module configured to, according to a function display sequence of the wearable device, determine a display sequence of a function icon corresponding to each set function in a line-of-sight range corresponding to the visual angle range information from the initial proportion control interface, and display a target function icon in the preset initial proportion control interface one by one according to the display sequence and the function icon; The first determining module is configured to identify a user's eye feature in a display process based on the wearable device, determine visual attention point information of the user in a visual angle range according to the user's eye feature, determine a first visual attention function icon corresponding to the visual attention point information according to the visual attention point information and the target function icon, and display the first visual attention function icon in the preset initial proportion control interface. The first determining module is configured to identify a user's eye feature in a display process based on the wearable device, determine visual attention point information of the user in a visual angle range according to the user's eye feature, determine a first visual attention function icon corresponding to the visual attention point information according to the visual attention point information and the target function icon, and display the first visual attention function icon in the preset initial proportion control interface. The first determining module is further configured to extract a pupil proportion feature of the pupil attention amplification image based on a preset feature convolution network, generate a pupil proportion feature corresponding to the pupil attention amplification image, obtain a set pupil proportion feature corresponding to the preset pupil focus image, generate a proportion amplification locking instruction in a case where the pupil proportion feature matches the set pupil proportion feature, and use the proportion amplification locking instruction to amplify the first visual attention function icon and lock the preset initial proportion control interface currently displayed; and display the target function icon of the preset initial proportion control interface one by one in a case where the pupil proportion feature does not match the set pupil proportion feature. The second determining module is configured to monitor a position of the visual attention point information in the preset initial proportion control interface in real time, display the first visual attention function icon in the preset initial proportion control interface in a reduced manner in response to a point corresponding to the visual attention point information leaving a visual area corresponding to the first visual attention function icon in the preset initial proportion control interface, and determine a second visual attention function icon based on the position of the visual attention point information in the preset initial proportion control interface, and display the second visual attention function icon in an amplified manner. The execution module is configured to generate a display proportion locking instruction to lock display content and a display proportion of a corresponding visual attention function icon in the preset initial proportion control interface in a case where the user's eye feature visually stays in the second visual attention function icon for more than a set time threshold, and generate a target proportion control interface.

6. The AR / VR / MR glasses function interface proportion control device according to claim 5, characterized in that, The device further includes a determination module configured to: obtain display position information of each visual attention function icon and target position information of the second visual attention function icon in the target proportion control interface; In a case where the dwell time of the visual position corresponding to the user eye feature on the position corresponding to the other visual attention function icon outside the target position information is greater than a set time threshold, a display proportion unlocking instruction is generated, the display proportion unlocking instruction being used to restore the display proportion of the visual attention function icon in the target proportion control interface to a set display proportion, and according to the target function icon in the preset initial proportion control interface, the one-by-one display of the subsequent function icons is sequentially performed.

7. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program, when executed by a processor, implements the steps of the method of any one of claims 1-4.

8. An electronic device, comprising: Comprise: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the executable instructions in the memory to implement the method of any one of claims 1-4.

Images