Head-mounted display equipment control method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a control method and device of head-mounted display equipment, the head-mounted display equipment and a storage medium, and belongs to the field of head-mounted display control. The method comprises the steps that the type of target display content is acquired in response to the fact that the motion trail of the control equipment is in an enclosed shape, and the target display content comprises display content in an area enclosed by the motion trail in a virtual display interface of the head-mounted display equipment; determining a target display parameter of the suspension layer according to the type of the target display content, and displaying the suspension layer on the virtual display interface in an overlapping manner according to the target display parameter; and adjusting the size of the target display content in the suspension layer. According to the embodiment of the invention, the control convenience and control efficiency of the head-mounted display equipment are improved.

Description

Technical Field

[0001] The present invention relates to the field of head mounted display control technology, and in particular to a control method and device for a head mounted display device, a head mounted display device and a storage medium. Background Art

[0002] With the rapid development of head-mounted display devices, head-mounted display devices have been widely used in many fields. At present, when a user needs to view the details of the displayed content in the display interface of the head-mounted display device, the user needs to go through cumbersome operations to adjust the size of the displayed content, resulting in low control convenience and efficiency of the display interface and poor user experience. Summary of the invention

[0003] Embodiments of the present invention provide a control method and apparatus for a head-mounted display device, a head-mounted display device, and a storage medium, aiming to improve the control convenience and control efficiency of the head-mounted display device.

[0004] In a first aspect, an embodiment of the present invention provides a method for controlling a head mounted display device, wherein the head mounted display device is communicatively connected to a control device, and the method includes:

[0005] In response to the motion trajectory of the control device being in an enclosed shape, acquiring a type of target display content, the target display content comprising display content within an area enclosed by the motion trajectory in a virtual display interface of the head mounted display device;

[0006] Determining target display parameters of the suspension layer according to the type of the target display content, and overlaying and displaying the suspension layer on the virtual display interface according to the target display parameters;

[0007] The size of the target display content in the floating layer is adjusted.

[0008] In a second aspect, an embodiment of the present invention further provides a control device for a head mounted display device, wherein the head mounted display device is communicatively connected to a control device, and the control device comprises:

[0009] a data acquisition module, configured to acquire a type of target display content in response to the motion trajectory of the control device being in an enclosed shape, wherein the target display content includes display content within an area enclosed by the motion trajectory in a virtual display interface of the head mounted display device;

[0010] A parameter determination module, used to determine the target display parameters of the floating layer according to the type of the target display content;

[0011] A floating layer display module, used for superimposing and displaying a floating layer on the virtual display interface according to the target display parameters;

[0012] The content adjustment module is used to adjust the size of the target display content in the floating layer.

[0013] In a third aspect, an embodiment of the present invention further provides a head-mounted display device, comprising a processor, a memory, a display device, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor, the memory, and the display device, wherein when the computer program is executed by the processor, the control method of the head-mounted display device as described in the first aspect is implemented.

[0014] In a fourth aspect, an embodiment of the present invention further provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the control method of the head-mounted display device as described in the first aspect.

[0015] The embodiments of the present invention provide a control method and device for a head-mounted display device, a head-mounted display device, and a storage medium. The embodiments of the present invention automatically determine target display parameters of a suspended layer according to the type of display content in an area enclosed by the motion trajectory of a control device in a virtual display interface of the head-mounted display device when the motion trajectory of the control device is in an enclosed shape, and overlay and display a corresponding suspended layer on the virtual display interface according to the target display parameters, and then adjust the size of the target display content in the suspended layer, thereby realizing a zoom-in operation or a zoom-out operation of the display content in the virtual display interface. During the whole process, the user only needs to simply operate the control device to draw a circle, and the operation is simple. In addition, the display parameters of the suspended layer can adapt to the type of display content to be adjusted in size, and do not require manual selection and configuration by the user. Therefore, the control convenience and control efficiency of the head-mounted display device are greatly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 is a schematic diagram of a scenario for implementing the control method of the head mounted display device provided by an embodiment of the present invention;

[0018] Figure 2 A schematic flow chart of a method for controlling a head mounted display device provided by an embodiment of the present invention;

[0019] Figure 3is a flow chart of another method for controlling a head mounted display device provided by an embodiment of the present invention;

[0020] Figure 4 is another schematic diagram of a scenario for implementing the control method of the head mounted display device provided in this embodiment;

[0021] Figure 5 A schematic block diagram of the structure of a control device for a head mounted display device provided by an embodiment of the present invention;

[0022] Figure 6 The present invention is a schematic structural block diagram of a head mounted display device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0023] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0024] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0025] It should be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0026] With the rapid development of head-mounted display devices, head-mounted display devices have been widely used in many fields. At present, when a user needs to view the details of the displayed content in the display interface of the head-mounted display device, the user needs to go through cumbersome operations to adjust the size of the displayed content, resulting in low control convenience and efficiency of the display interface and poor user experience.

[0027] To solve the above problems, the embodiments of the present invention provide a control method, device, head-mounted display device and storage medium for a head-mounted display device. The control method automatically determines the target display parameters of the suspension layer according to the type of display content in the area enclosed by the motion trajectory in the virtual display interface of the head-mounted display device when the motion trajectory of the control device is in an enclosed shape, and superimposes and displays the corresponding suspension layer on the virtual display interface according to the target display parameters, and then adjusts the size of the target display content in the suspension layer, thereby realizing the zooming in or zooming out of the display content in the virtual display interface. During the whole process, the user only needs to simply operate the control device to draw a circle, which is simple to operate, and the display parameters of the suspension layer can adapt to the type of display content to be adjusted in size, without the need for the user to manually select and configure, thereby greatly improving the control convenience and control efficiency of the head-mounted display device.

[0028] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0029] See also Figure 1 , Figure 1 It is a schematic diagram of a scenario for implementing the control method of the head mounted display device provided by an embodiment of the present invention.

[0030] like Figure 1 As shown, the scene includes a head mounted display device 100 and a control device 200, and the head mounted display device 100 and the control device 200 are in communication connection. Among them, the head mounted display device 100 and the control device 200 can establish a communication connection through Bluetooth or mobile wireless interconnection (wireless fidelity, wifi) and the like. The control device 200 includes a handheld device or a wearable device. For example, the handheld device may include a smart phone, a tablet computer or a remote control device, and the wearable device may include a finger ring device, an arm ring device, a wristband device or a necklace device. For example, the finger ring device is worn on the user's finger, and the user can control the movement of the control device 200 by moving the finger. The head mounted display device 100 may include augmented reality (AR) glasses, AR helmets, mixed reality (MR) glasses and MR helmets, etc.

[0031] In some embodiments, the control device 200 obtains its own motion information and sends the motion information to the head-mounted display device 100; the head-mounted display device 100 determines the motion trajectory of the control device 200 according to the motion information; in response to the motion trajectory of the control device 200 being enclosed, the type of target display content is obtained, and the target display content includes the display content in the area enclosed by the motion trajectory in the virtual display interface of the head-mounted display device 100; according to the type of target display content, the target display parameter of the suspension layer is determined, and according to the target display parameter, the suspension layer is superimposed on the virtual display interface; the size of the target display content in the suspension layer is adjusted. Among them, the control device 200 includes an inertial measurement unit (IMU), and the motion information of the control device 200 itself can be obtained through the IMU. In other embodiments, the control device 200 can also first determine its own motion trajectory according to its own motion information, and then send its own motion trajectory to the head-mounted display device 100 to reduce the consumption of computing resources of the head-mounted display device 100.

[0032] The following will be combined Figure 1 The control method of the head mounted display device provided by the embodiment of the present invention is described in detail in the scenario in FIG. Figure 1 The scenario is only used to explain the control method of the head-mounted display device provided by the embodiment of the present invention, but does not constitute a limitation on the application scenario of the control method of the head-mounted display device provided by the embodiment of the present invention.

[0033] See also Figure 2 , Figure 2 It is a flowchart of a control method of a head mounted display device provided by an embodiment of the present invention.

[0034] like Figure 1 As shown, the control method of the head mounted display device includes steps S101 to S103.

[0035] Step S101: In response to a motion trajectory of a control device being in an enclosed shape, obtaining a type of target display content, where the target display content includes display content within an area enclosed by the motion trajectory in a virtual display interface of a head mounted display device.

[0036] In this embodiment, the motion trajectory of the control device is enclosed, which means that the figure formed by the motion trajectory of the control device is a closed figure, for example, the figure formed by the motion trajectory of the control device is a circle, an ellipse or a rectangle, etc. It can be understood that the motion trajectory of the control device is enclosed, that is, the motion trajectory of the control device is a closed figure.

[0037] In some embodiments, the control device obtains its own motion information through a built-in IMU and sends the motion information to the head-mounted display device, and the head-mounted display device determines the motion trajectory of the control device based on the motion information. Alternatively, the control device obtains its own motion information through a built-in IMU, determines its own motion trajectory based on the motion information, and then sends its own motion trajectory to the head-mounted display device.

[0038] In some embodiments, in response to the motion trajectory of the control device being in an enclosed shape, obtaining the type of target display content may include: in response to the motion trajectory of the control device being in an enclosed shape, and the motion angular velocity of the control device being greater than or equal to a preset threshold, obtaining the type of target display content. The preset threshold can be set based on actual conditions, and the embodiment of the present invention does not specifically limit this. For example, the preset threshold is 15 degrees / second or 30 degrees / second. This embodiment can avoid user misoperation and improve the accuracy of the operation of triggering the zoom operation of the displayed content.

[0039] In some embodiments, in response to the motion trajectory of the control device being in an enclosed shape, obtaining the type of the target display content may include: in response to the motion trajectory of the control device being in an enclosed shape, and the area of ​​the area enclosed by the motion trajectory is greater than or equal to the first area and smaller than the second area, obtaining the type of the target display content. The first area is smaller than the second area, and the second area is the total area of ​​the virtual display interface of the head-mounted display device. This embodiment can prevent the user from operating the control device to draw a circle that is too small or too large, effectively avoid user misoperation, and improve the accuracy of the operation of triggering the size adjustment of the display content.

[0040] Step S102: Determine target display parameters of the floating layer according to the type of target display content, and overlay and display the floating layer on the virtual display interface according to the target display parameters.

[0041] In this embodiment, the type of target display content may include text, pictures and / or videos, etc., and the target display parameters of the floating layer include the target display size and / or target display position of the floating layer. In this embodiment, the floating layer is superimposed on the virtual display interface using display parameters that match the type of display content to be magnified, so that the display size and / or display position of the floating layer better meets the usage requirements of the display content to be magnified, and the user experience is better.

[0042] In some embodiments, the target display parameters of the suspension layer include the target display position of the suspension layer. According to the type of the target display content, determining the target display parameters of the suspension layer may include: in response to the type of the target display content being a preset type, determining the first display position as the target display position of the suspension layer, the first display position being the position of the area enclosed by the motion track in the virtual display interface of the head-mounted display device. Among them, the preset type can be set based on actual conditions, and the embodiments of the present invention do not specifically limit this. For example, the preset type is text. In this embodiment, the suspension layer is superimposed on the position of the area enclosed by the motion track in the virtual display interface of the head-mounted display device, so that the position of the suspension layer is consistent with the position of the display content to be enlarged, which is more convenient for users to read.

[0043] In some embodiments, the target display parameters of the floating layer include the target display position of the floating layer. According to the type of the target display content, determining the target display parameters of the floating layer may include: in response to the type of the target display content not being a preset type, determining the second display position as the target display position of the floating layer, the second display position including the display position other than the first display position in the virtual display interface. This embodiment can avoid the blocking of the target display content by the floating layer by superimposing the floating layer at a position other than the position where the area enclosed by the motion track is located in the virtual display interface of the head-mounted display device, so that the user can simultaneously read the target display content of the original size and the target display content after the resize, and the user experience is better.

[0044] In some embodiments, the second display position includes a central display position of a virtual display interface of the head mounted display device or a display position adjacent to the first display position. In this embodiment, by superimposing a floating layer on the central display position of the virtual display interface or a display position adjacent to the first display position, the user can better read the target display content, and the user experience is better.

[0045] In some embodiments, the target display parameters of the floating layer include the target display size of the floating layer. According to the type of target display content, determining the target display parameters of the floating layer may include: according to the type of target display content, querying a pre-stored mapping relationship table between the type of display content and the display size of the floating layer to obtain the target display size of the floating layer. Among them, the mapping relationship table between the pre-stored type of display content and the display size of the floating layer is established in advance based on experience, and this embodiment does not specifically limit this. For example, the types of display content include text, pictures, and videos. Text corresponds to display size A, pictures correspond to display size B, and videos correspond to display size C. Display size A is smaller than display size B, and display size B is smaller than or equal to display size C.

[0046] In some embodiments, the target display parameters of the suspension layer include the target display size of the suspension layer. According to the type of target display content, determining the target display parameters of the suspension layer may include: determining the first display size of the suspension layer according to the type of target display content; obtaining the second display size of the area enclosed by the motion track in the virtual display interface of the head-mounted display device; when the first display size is greater than or equal to the second display size, determining the first display size as the target display size of the suspension layer; when the first display size is less than the second display size, determining the second display size as the target display size of the suspension layer. This embodiment can ensure that the display size of the suspension layer is greater than or equal to the display size of the area enclosed by the motion track in the virtual display interface of the head-mounted display device, so that the suspension layer can more comprehensively adjust the size of the displayed content, and the user experience is better.

[0047] In some embodiments, the target display parameters of the floating layer include the target display size and / or target display position of the floating layer. According to the target display parameters, superimposing and displaying the floating layer on the virtual display interface may include: superimposing and displaying the floating layer on the target display position in the virtual display interface; or, superimposing and displaying the floating layer of the target display size on the virtual display interface; or, superimposing and displaying the floating layer of the target display size on the target display position in the virtual display interface.

[0048] In some embodiments, in response to the target display position being the position of an area enclosed by a motion trajectory in a virtual display interface of a head-mounted display device, a fully transparent floating layer is superimposed and displayed on the target display position in the virtual display interface, or a fully transparent floating layer of a target display size is superimposed and displayed on the target display position in the virtual display interface; in response to the target display position not being the position of an area enclosed by a motion trajectory in the virtual display interface of the head-mounted display device, an opaque floating layer is superimposed and displayed on the target display position in the virtual display interface, or an opaque floating layer of a target display size is superimposed and displayed on the target display position in the virtual display interface.

[0049] In some embodiments, the shape of the suspension layer is a regular shape, and the shape of the suspension layer is substantially the same as the shape of the enclosed motion track. Since the motion track of the control device is usually generated by the movement of the user's fingers, arms, palms, etc., it is impossible to ensure that the shape of the motion track of the control device is regular. However, this embodiment can display a suspension layer with a shape that is substantially the same as the shape of the motion track and is regular, which makes it easier for users to circle the display content that needs to be resized, and provides a better user experience.

[0050] Step S103: adjusting the size of the target display content in the floating layer.

[0051] In this embodiment, adjusting the size of the target display content in the floating layer may include zooming in or out the target display content in the floating layer. The target display content in the floating layer may be zoomed in by a default zoom factor, or the target display content in the floating layer may be zoomed out by a default zoom factor. For example, if the default zoom factor is 2, the target display content in the floating layer is zoomed in by 2. For another example, if the default zoom factor is 1.5, the target display content in the floating layer is zoomed out by 1.5.

[0052] In some embodiments, magnifying the target display content in the floating layer may include: obtaining a target magnification factor, which is related to the eye diopter of the user wearing the head-mounted display device and / or the type of the target display content; and magnifying the target display content in the floating layer according to the target magnification factor. In this embodiment, the target display content in the floating layer is magnified according to the target magnification factor related to the eye diopter of the user wearing the head-mounted display device and / or the type of the target display content, so that the magnified target display content is more suitable for the user to read, further improving the user experience.

[0053] In some embodiments, obtaining the target magnification may include: determining the adjustment value of the magnification according to the eye diopter of the user wearing the head-mounted display device, and adding the adjustment value of the magnification to the preset magnification to obtain the target magnification. Among them, the eye diopter of the user wearing the head-mounted display device can be manually input by the user, or can be measured by the head-mounted display device. The preset magnification can be set based on actual conditions, and the embodiment of the present invention does not specifically limit this. For example, the user's eye diopter F = 4.00 (400 degrees myopia), the preset magnification is 2 times, and the adjustment value of the magnification is D = 1 / F = 1 / 4, then the target magnification is 2.25 times. This embodiment can adjust the default magnification to make the enlarged display content adapt to the user's eye diopter when the user does not select myopia or hyperopia lens accessories, so as to facilitate the user's viewing.

[0054] In some embodiments, obtaining the target magnification may include: determining the target magnification based on the eye refractive power of the user wearing the head-mounted display device; or determining the target magnification based on the type of target display content; or determining the target magnification based on the eye refractive power of the user wearing the head-mounted display device and the type of target display content.

[0055] In some embodiments, determining the target magnification according to the eye diopter of the user wearing the head-mounted display device may include: according to the eye diopter of the user wearing the head-mounted display device, querying a pre-stored mapping relationship table between the eye diopter and the magnification to obtain the target magnification. Among them, the mapping relationship table between the eye diopter and the magnification is established in advance based on experience, and the embodiment of the present invention does not specifically limit this. This embodiment can magnify the displayed content by determining the adapted target magnification when the user has not selected myopia or hyperopia lens accessories, so that the magnified displayed content can be adapted to the user's eye diopter, which is convenient for the user to watch.

[0056] In some embodiments, according to the eye diopter of the user wearing the head mounted display device, determining the target magnification may include: subtracting the diopter of the eyeglasses of the head mounted display device from the eye diopter of the user wearing the head mounted display device to obtain a diopter difference; according to the diopter difference, querying a pre-stored mapping relationship table between the diopter difference and the magnification to obtain the target magnification. Among them, the pre-stored mapping relationship table between the diopter difference and the magnification is established in advance based on experience, and the embodiment of the present invention does not specifically limit this. Due to the change in the user's eye diopter, the myopia or hyperopia lens accessories previously selected by the user do not meet the user's vision, so that the user cannot read the displayed content well. In this embodiment, the difference between the user's eye diopter and the diopter of the eyeglasses of the head mounted display device is used to determine the adapted target magnification to magnify the displayed content, so that the enlarged displayed content can be adapted to the user's eye diopter, which is convenient for the user to watch, and does not require the user to change the eyeglasses, and the user experience is better.

[0057] In some embodiments, according to the type of the target display content, determining the target magnification may include: in response to the type of the target display content not being text, determining the preset magnification as the target magnification; or in response to the type of the target display content being text, obtaining the current font size of the target display content; determining the target magnification according to the current font size of the target display content and the preset font size of the head-mounted display device. The preset font size is related to the clarity of the screen of the head-mounted display device, and can be set based on actual conditions, and the embodiments of the present invention do not specifically limit this. For example, the screen of the head-mounted display device requires the font size of the text to be 24 to achieve readability, that is, the preset font size is 24, and the current font size of the target display content is 8, therefore, the target magnification of the target display content is 8 / 24=3 times.

[0058] In some embodiments, determining the target magnification according to the eye diopter of the user wearing the head-mounted display device and the type of the target display content may include: determining a first magnification according to the eye diopter of the user wearing the head-mounted display device; determining a second magnification according to the type of the target display content; and determining the larger of the first magnification and the second magnification as the target magnification. The first magnification may be obtained by querying a pre-stored mapping relationship table between the eye diopter and the magnification according to the eye diopter of the user wearing the head-mounted display device, or by determining an adjustment value of the magnification according to the eye diopter of the user wearing the head-mounted display device, and the first magnification is obtained by adding the adjustment value of the magnification to the preset magnification.

[0059] In some embodiments, Figure 3 As shown, after step S103, the following steps are also included:

[0060] Step S104: In response to a magnification adjustment operation or a reduction adjustment operation of the control device, the size of the target display content in the floating layer is adjusted.

[0061] In this embodiment, after adjusting the size of the target display content in the floating layer, the user can adjust the size of the target display content in the floating layer by manipulating the control device according to his / her needs, which provides a better user experience.

[0062] In some embodiments, the control device includes a touch bar, and in response to the magnification adjustment operation of the control device, adjusting the size of the target display content in the floating layer may include: in response to the first sliding operation of the control device on the touch bar, lowering the magnification of the target display content in the floating layer by a first unit magnification, and adjusting the size of the target display content in the floating layer according to the lowered magnification; or, in response to the second sliding operation of the control device on the touch bar, raising the magnification of the target display content in the floating layer by a second unit magnification, and adjusting the size of the target display content in the floating layer according to the increased magnification. Wherein, the first sliding operation is different from the second sliding operation, and the first unit magnification and the second unit magnification may be the same or different. For example, the first unit magnification and the second unit magnification are both 0.5 times, or the first unit magnification is 0.25 times and the second unit magnification is 0.5 times.

[0063] In some embodiments, the first sliding operation includes a sliding operation of the user's finger on the touch bar from top to bottom, and the second sliding operation includes a sliding operation of the user's finger on the touch bar from bottom to top. Figure 4As shown, in response to the user's finger sliding operation on the touch bar from top to bottom, the control device 200 sends a magnification reduction instruction to the head-mounted display device 100. According to the magnification reduction instruction, the head-mounted display device 100 reduces the magnification of the target display content in the suspended layer by a first unit magnification, and adjusts the size of the target display content in the suspended layer according to the reduced magnification. In response to the user's finger sliding operation on the touch bar from bottom to top, the control device 200 sends a magnification increase instruction to the head-mounted display device 100. According to the magnification increase instruction, the head-mounted display device 100 increases the magnification of the target display content in the suspended layer by a second unit magnification, and adjusts the size of the target display content in the suspended layer according to the increased magnification.

[0064] In some embodiments, the control device includes a first button and a second button. In response to a magnification adjustment operation of the control device, adjusting the size of the target display content in the floating layer may include: in response to a click operation of the control device on the first button, lowering the magnification of the target display content in the floating layer by a first unit magnification, and adjusting the size of the target display content in the floating layer according to the lowered magnification; or, in response to a click operation of the control device on the second button, increasing the magnification of the target display content in the floating layer by a second unit magnification, and adjusting the size of the target display content in the floating layer according to the increased magnification.

[0065] In some embodiments, the control device includes a touch bar, and in response to the control device's zoom-out magnification adjustment operation, adjusting the size of the target display content in the floating layer may include: in response to the control device's third sliding operation on the touch bar, lowering the zoom-out magnification of the target display content in the floating layer by a third unit magnification, and adjusting the size of the target display content in the floating layer according to the lowered zoom-out magnification; or, in response to the control device's fourth sliding operation on the touch bar, raising the zoom-out magnification of the target display content in the floating layer by a fourth unit magnification, and adjusting the size of the target display content in the floating layer according to the increased zoom-out magnification. The third sliding operation is different from the fourth sliding operation, and the third unit magnification and the fourth unit magnification may be the same or different. For example, the third unit magnification and the fourth unit magnification are both 0.25 times, or the third unit magnification is 0.25 times and the second unit magnification is 0.5 times.

[0066] In some embodiments, the third sliding operation includes a sliding operation of the user's finger on the touch bar from the center to the sides (outward sliding operation), and the fourth sliding operation includes a sliding operation of the user's finger on the touch bar from the sides to the center (inward sliding operation). Alternatively, the third sliding operation includes a sliding operation of the user's finger on the touch bar from top to bottom, and the fourth sliding operation includes a sliding operation of the user's finger on the touch bar from bottom to top.

[0067] In some embodiments, after step S103, it also includes: in response to the floating layer being in a selected state, obtaining the motion information of the control device; adjusting the display position of the floating layer on the virtual display interface according to the motion information, and adjusting the size of the current display content in the floating layer after the display position is adjusted. In this embodiment, after selecting the floating layer, the position of the floating layer on the display interface is synchronously adjusted based on the movement of the control device to change the enlarged display content in the floating layer. The whole process only requires the user to simply operate the control device to achieve the movement. The operation is simple, which is convenient for the user to move or move the floating layer at will, greatly improving the user experience.

[0068] In some embodiments, the control method further includes: in response to a long-term gaze operation of the user wearing the head-mounted display device on the suspended layer, switching the current state of the suspended layer being gazed to a selected state, and changing the visual performance of the suspended layer to indicate that the suspended layer is in a selected state. Among them, the long-term gaze operation of the user's eyeball on the suspended layer refers to an operation in which the user's eyeball gazes on the suspended layer for a time period greater than or equal to a preset time period. The preset time period can be set based on actual conditions, and the embodiments of the present invention do not specifically limit this. For example, the preset time period is 3 seconds.

[0069] In some embodiments, the motion information of the control device includes the motion direction and motion distance of the control device. According to the motion information, adjusting the display position of the suspended layer on the virtual display interface may include: determining the moving direction and moving distance of the suspended layer according to the motion direction and the motion distance; and adjusting the display position of the suspended layer on the virtual display interface according to the moving direction and the moving distance of the suspended layer. For example, the motion direction of the control device is determined as the moving direction of the suspended layer, and the motion distance of the control device is converted into the moving distance of the suspended layer according to a preset conversion relationship. Among them, the preset conversion relationship can be set based on actual conditions, and the embodiments of the present invention do not specifically limit this.

[0070] For example, if the movement direction of the suspension layer is above the suspension layer, and the movement distance of the suspension layer is 2 cm, the suspension layer is moved upward by 2 cm as a whole, thereby changing the position of the suspension layer on the display screen. It is understandable that after adjusting the position of the suspension layer on the display screen, the size of the display content in the suspension layer after the adjustment is synchronously adjusted. Among them, the target magnification of the current display content in the suspension layer after the adjustment can be automatically adjusted according to the type of the current display content, or the magnification used by the suspension layer before the adjustment can be used, or a preset magnification can be used, and the embodiment of the present invention does not specifically limit this.

[0071] In some embodiments, after step S103, the process further includes: in response to a user triggering a return button on the control device, deleting the floating layer and restoring the magnified display content in the virtual display interface. The user triggering the return button on the control device may include any one of a click operation, a double-click operation, and a long press operation of the return button on the control device.

[0072] In some embodiments, the size of the target display content in the floating layer is adjusted, including: in response to the movement direction of the control device being a first direction, the target display content in the floating layer is enlarged; or, in response to the movement direction of the control device being a second direction, the target display content in the floating layer is reduced. The first direction is one of the clockwise direction and the counterclockwise direction, and the second direction is the other of the clockwise direction and the counterclockwise direction. For example, the user operates the control device to quickly draw a circle in a clockwise direction, thereby triggering a zoom-in operation, and then superimposing the floating layer on the virtual display interface, and zooming in the target display content in the floating window in the virtual display interface. For another example, the user operates the control device to quickly draw a circle in a counterclockwise direction, thereby triggering a zoom-out operation, and then superimposing the floating layer on the virtual display interface, and zooming in the target display content in the floating window in the virtual display interface.

[0073] In some embodiments, reducing the target display content in the floating layer may include: reducing the target display content in the floating layer according to a preset reduction ratio, or reducing the target display content in the floating layer according to a target reduction ratio, where the target reduction ratio is related to the type of the target display content.

[0074] In some embodiments, the control method of the head-mounted display device further includes: in response to the motion trajectory of the control device being in an enclosed shape and the motion direction of the control device being a first direction, obtaining the type of target display content, the target display content including the display content in the area enclosed by the motion trajectory in the virtual display interface of the head-mounted display device; determining the target display parameters of the floating layer according to the type of the target display content, and superimposing and displaying the floating layer on the virtual display interface according to the target display parameters; amplifying the target display content in the floating layer; or in response to the motion trajectory of the control device being in an enclosed shape and the motion direction of the control device being a second direction, reducing all display content in the virtual display interface according to a preset reduction ratio. The first direction is one of the clockwise direction and the counterclockwise direction, and the second direction is the other of the clockwise direction and the counterclockwise direction.

[0075] In some embodiments, the control method of the head-mounted display device further includes: in response to the motion trajectory of the control device being in an enclosed shape, and the area of ​​the area enclosed by the motion trajectory being smaller than a preset area, obtaining the type of target display content, the target display content including the display content in the area enclosed by the motion trajectory in the virtual display interface of the head-mounted display device; determining the target display parameters of the suspension layer according to the type of the target display content, and superimposing the suspension layer on the virtual display interface according to the target display parameters; amplifying the target display content in the suspension layer; or in response to the motion trajectory of the control device being in an enclosed shape, and the area of ​​the area enclosed by the motion trajectory being greater than or equal to the preset area, reducing all display content in the virtual display interface according to a preset reduction ratio. Wherein, the preset area is the total area of ​​the virtual display interface of the head-mounted display device.

[0076] See also Figure 5 , Figure 5 The present invention provides a schematic block diagram of a control device for a head mounted display device provided in an embodiment of the present invention. The control device is applied to the head mounted display device, and the head mounted display device is connected to the control device for communication.

[0077] like Figure 5 As shown, the control device 110 of the head mounted display device includes:

[0078] A data acquisition module 111, configured to acquire a type of target display content in response to the motion trajectory of the control device being in an enclosed shape, wherein the target display content includes display content in an area enclosed by the motion trajectory in a virtual display interface of the head mounted display device;

[0079] A parameter determination module 112, configured to determine a target display parameter of a floating layer according to the type of the target display content;

[0080] A floating layer display module 113, configured to overlay and display a floating layer on the virtual display interface according to the target display parameters;

[0081] The content adjustment module 114 is used to adjust the size of the target display content in the floating layer.

[0082] In some embodiments, the content adjustment module 114 is further configured to:

[0083] Acquire a target magnification, where the target magnification is related to the eye diopter of a user wearing the head mounted display device and / or the type of the target display content;

[0084] The size of the target display content in the floating layer is adjusted according to the target magnification.

[0085] In some embodiments, the content adjustment module 114 is further configured to:

[0086] In response to the type of the target display content being text, obtaining a current font size of the target display content;

[0087] A target magnification factor is determined according to a current font size of the target display content and a preset font size of the head mounted display device.

[0088] In some embodiments, the target display parameter includes a target display position of the floating layer, and the parameter determination module 112 is further configured to:

[0089] In response to the type of the target display content being a preset type, determining a first display position as the target display position, the first display position being a position where an area enclosed by the motion track in the virtual display interface is located; or,

[0090] In response to the type of the target display content not being a preset type, a second display position is determined as the target display position, where the second display position includes a display position in the virtual display interface other than the first display position.

[0091] In some embodiments, the target display parameter includes a target display size of the floating layer, and the parameter determination module 112 is further configured to:

[0092] According to the type of the target display content, a pre-stored mapping relationship table between the type of display content and the display size of the floating layer is queried to obtain the target display size of the floating layer.

[0093] In some embodiments, the content adjustment module 114 is further configured to:

[0094] In response to the magnification adjustment operation of the control device, the size of the target display content in the floating layer is adjusted.

[0095] In some embodiments, the control device 110 of the head mounted display device further includes:

[0096] The data acquisition module 111 is further used to acquire the motion information of the control device in response to the suspension layer being in the selected state;

[0097] A floating layer adjustment module, used to adjust the display position of the floating layer on the virtual display interface according to the motion information;

[0098] The content adjustment module 114 is further configured to adjust the size of the currently displayed content in the floating layer after the display position is adjusted.

[0099] It should be noted that technicians in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of the control device of the head-mounted display device described above can refer to the corresponding process in the aforementioned control method embodiment of the head-mounted display device, and will not be repeated here.

[0100] See also Figure 6 , Figure 6 It is a schematic block diagram of the structure of a head mounted display device provided by an embodiment of the present invention.

[0101] like Figure 6 As shown, the head mounted display device 100 includes a processor 101, a memory 102 and a display device 103, and the processor 101, the memory 102 and the display device 103 are connected via a bus 104, such as an I2C (Inter-integrated Circuit) bus. The head mounted display device 100 may include augmented reality (AR) glasses, virtual reality (VR) glasses, mixed reality (MR) glasses, AR helmets, VR helmets, MR helmets and other devices.

[0102] Specifically, the processor 101 is used to provide computing and control capabilities to support the operation of the entire head-mounted display device. The processor 101 can be a central processing unit (CPU), and the processor 101 can also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0103] Specifically, the memory 102 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0104] Those skilled in the art will understand that Figure 6 The structure shown in the figure is only a block diagram of a partial structure related to the embodiment of the present invention, and does not constitute a limitation on the head-mounted display device to which the embodiment of the present invention is applied. The specific head-mounted display device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0105] The processor 101 is used to run a computer program stored in the memory 102, and implement any one of the control methods for the head mounted display device provided by the embodiments of the present invention when executing the computer program.

[0106] In some embodiments, the processor 101 is used to run a computer program stored in the memory, and implement the following steps when executing the computer program:

[0107] In response to the motion trajectory of the control device being in an enclosed shape, acquiring a type of target display content, the target display content comprising display content within an area enclosed by the motion trajectory in a virtual display interface of the head mounted display device;

[0108] Determining target display parameters of the suspension layer according to the type of the target display content, and overlaying and displaying the suspension layer on the virtual display interface according to the target display parameters;

[0109] The size of the target display content in the floating layer is adjusted.

[0110] In some embodiments, when the processor 101 adjusts the size of the target display content in the floating layer, it is used to implement:

[0111] Acquire a target magnification, where the target magnification is related to the eye diopter of a user wearing the head mounted display device and / or the type of the target display content;

[0112] The target display content in the floating layer is magnified according to the target magnification factor.

[0113] In some embodiments, the processor 101 is used to implement the following when acquiring the target magnification:

[0114] In response to the type of the target display content being text, obtaining a current font size of the target display content;

[0115] A target magnification factor is determined according to a current font size of the target display content and a preset font size of the head mounted display device.

[0116] In some embodiments, the target display parameter includes a target display position of the floating layer. When the processor 101 determines the target display parameter of the floating layer according to the type of the target display content, it is configured to implement:

[0117] In response to the type of the target display content being a preset type, determining a first display position as the target display position, the first display position being a position where an area enclosed by the motion track in the virtual display interface is located; or,

[0118] In response to the type of the target display content not being a preset type, a second display position is determined as the target display position, where the second display position includes a display position in the virtual display interface other than the first display position.

[0119] In some embodiments, the target display parameter includes a target display size of the floating layer. When the processor 101 determines the target display parameter of the floating layer according to the type of the target display content, it is configured to implement:

[0120] According to the type of the target display content, a pre-stored mapping relationship table between the type of display content and the display size of the floating layer is queried to obtain the target display size of the floating layer.

[0121] In some embodiments, after adjusting the size of the target display content in the floating layer, the processor 101 is further configured to implement:

[0122] In response to a magnification adjustment operation or a reduction adjustment operation of the control device, a size of the target display content in the floating layer is adjusted.

[0123] In some embodiments, after adjusting the size of the target display content in the floating layer, the processor 101 is further configured to implement:

[0124] In response to the suspension layer being in a selected state, acquiring motion information of the control device;

[0125] According to the motion information, the display position of the floating layer on the virtual display interface is adjusted, and the size of the current display content in the floating layer after the display position is adjusted is adjusted.

[0126] It should be noted that those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the head-mounted display device described above can refer to the corresponding process in the aforementioned control method embodiment of the head-mounted display device, and will not be repeated here.

[0127] An embodiment of the present invention further provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any method for controlling a head-mounted display device provided in the description of the embodiment of the present invention.

[0128] The storage medium may be an internal storage unit of the head mounted display device described in the foregoing embodiment, such as a hard disk or memory of the head mounted display device. The storage medium may also be an external storage device of the head mounted display device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc., provided on the head mounted display device.

[0129] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0130] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0131] The serial numbers of the embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for controlling a head mounted display device, characterized in that: The head mounted display device is communicatively connected with a control device, and the method comprises: In response to the motion trajectory of the control device being in an enclosed shape, acquiring a type of target display content, the target display content comprising display content within an area enclosed by the motion trajectory in a virtual display interface of the head mounted display device; Determining target display parameters of the suspension layer according to the type of the target display content, and overlaying and displaying the suspension layer on the virtual display interface according to the target display parameters; The size of the target display content in the floating layer is adjusted.

2. The control method according to claim 1, characterized in that: The adjusting the size of the target display content in the floating layer includes: Acquire a target magnification, where the target magnification is related to the eye diopter of a user wearing the head mounted display device and / or the type of the target display content; The target display content in the floating layer is magnified according to the target magnification factor.

3. The control method according to claim 2, characterized in that: The obtaining of the target magnification comprises: In response to the type of the target display content being text, obtaining a current font size of the target display content; A target magnification factor is determined according to a current font size of the target display content and a preset font size of the head mounted display device.

4. The control method according to claim 1, characterized in that: The target display parameter includes a target display position of the floating layer, and determining the target display parameter of the floating layer according to the type of the target display content includes: In response to the type of the target display content being a preset type, determining a first display position as the target display position, the first display position being a position of an area enclosed by the motion track in the virtual display interface; or, In response to the type of the target display content not being a preset type, a second display position is determined as the target display position, where the second display position includes a display position in the virtual display interface other than the first display position.

5. The control method according to claim 1, characterized in that: The target display parameter includes a target display size of the floating layer, and determining the target display parameter of the floating layer according to the type of the target display content includes: According to the type of the target display content, a pre-stored mapping relationship table between the type of display content and the display size of the floating layer is queried to obtain the target display size of the floating layer.

6. The control method according to claim 1, characterized in that: After adjusting the size of the target display content in the floating layer, the method further includes: In response to a magnification adjustment operation or a reduction adjustment operation of the control device, a size of the target display content in the floating layer is adjusted.

7. The control method according to any one of claims 1 to 6, characterized in that: After adjusting the size of the target display content in the floating layer, the method further includes: In response to the suspension layer being in a selected state, acquiring motion information of the control device; According to the motion information, the display position of the floating layer on the virtual display interface is adjusted, and the size of the current display content in the floating layer after the display position is adjusted is adjusted.

8. A control device for a head mounted display device, characterized in that: The head mounted display device is communicatively connected with a control device, and the control device comprises: a data acquisition module, configured to acquire a type of target display content in response to the motion trajectory of the control device being in an enclosed shape, wherein the target display content includes display content within an area enclosed by the motion trajectory in a virtual display interface of the head mounted display device; A parameter determination module, used to determine the target display parameters of the floating layer according to the type of the target display content; A floating layer display module, used for superimposing and displaying a floating layer on the virtual display interface according to the target display parameters; The content adjustment module is used to adjust the size of the target display content in the floating layer.

9. A head mounted display device, characterized in that: The head-mounted display device includes a processor, a memory, a display device, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor, the memory and the display device, wherein when the computer program is executed by the processor, the control method of the head-mounted display device according to any one of claims 1 to 8 is realized.

10. A storage medium for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the control method of the head-mounted display device according to any one of claims 1 to 8.

Images