Head-mounted device control method and device, electronic device and storage medium

Abstract

The embodiment of the invention provides a control method and device of a head-mounted device, the head-mounted device and a storage medium, and the method comprises the steps: responding to a condition that the head-mounted device is in a wearing state, and determining a sight line activity region of a user of the head-mounted device; determining a display area of a screen of the head-mounted equipment; and outputting prompt information in response to the situation that the sight line activity area is outside the display area, so as to prompt the user to adjust the wearing parameters of the head-mounted equipment through the prompt information, thereby reducing the risk that the head-mounted equipment falls off when the user wears the head-mounted equipment, and improving the use safety of the head-mounted equipment.

Description

Technical Field

[0001] The present invention relates to the technical field of head-mounted devices, and in particular to a control method, device, electronic device and storage medium for a head-mounted device. Background Art

[0002] With the continuous improvement of communication technology, more and more smart devices have emerged. Smart glasses are one of them. As a kind of head-mounted device, smart glasses have similar functions to smart phones. They can control taking pictures, video calls and identifying directions through voice, as well as surfing the Internet, processing text messages and emails, etc.

[0003] When users use smart glasses, the user's wearing status has a great impact on the user's ability to view the content displayed on the smart glasses. For example, when the user's wearing posture is incorrect, the user may not be able to view the content displayed on the smart glasses screen. In addition, if the user wears the smart glasses incorrectly, the smart glasses may easily fall and be damaged. Summary of the invention

[0004] The embodiments of the present invention provide a control method, device, electronic device and storage medium for a head-mounted device, aiming to reduce the risk of the head-mounted device falling when a user wears the head-mounted device, thereby improving the safety of the head-mounted device.

[0005] In a first aspect, an embodiment of the present invention provides a method for controlling a head mounted device, the method comprising:

[0006] In response to the head mounted device being in a wearing state, determining a visual activity area of ​​a user of the head mounted device;

[0007] Determining a display area of ​​the head mounted device screen;

[0008] In response to the sight line activity area being outside the display area, outputting prompt information to prompt the user to adjust the wearing parameters of the head mounted device through the prompt information.

[0009] Optionally, in response to the sight line activity area being outside the display area, outputting prompt information to prompt the user to adjust the wearing parameters of the head mounted device through the prompt information, includes:

[0010] In response to the sight line activity area being outside the display area, determining a target number of times the sight line activity area is outside the display area within a first preset time period;

[0011] If the target number of times is greater than the preset number of times, a prompt message is output to prompt the user to adjust the wearing parameters of the head mounted device through the prompt message.

[0012] Optionally, in response to the head mounted device being in a wearing state, determining a visual activity area of ​​a user of the head mounted device includes:

[0013] In response to the head mounted device being in a wearing state, when it is detected that the eyeball position of the user of the head mounted device is in a changing state, determining an intersection point between the vertical line corresponding to the eyeball and the screen;

[0014] The intersection points corresponding to the second preset time length are obtained, and the visual line activity area of ​​the user is determined according to the intersection points corresponding to the second preset time length.

[0015] Optionally, the acquiring the intersection point corresponding to the second preset time length, and determining the visual activity area of ​​the user according to the intersection point corresponding to the second preset time length, includes:

[0016] Acquire the intersection points corresponding to the second preset time length, and determine the intersection area of ​​the intersection points corresponding to the second preset time length, wherein all the intersection points corresponding to the second preset time length exist in the intersection area;

[0017] The intersection area of ​​the intersection points corresponding to the second preset time length is determined as the visual line activity area of ​​the user.

[0018] Optionally, the method comprises:

[0019] Establishing a coordinate system with the display area of ​​the head mounted device, determining vertex coordinates of four vertices corresponding to the display area in the coordinate system, and determining intersection coordinates of each intersection within the second preset time length in the coordinate system;

[0020] The relative position relationship between the sight line activity area and the display area is determined according to the vertex coordinates and the intersection coordinates, wherein the relative position relationship includes the sight line activity area being outside the display area and the sight line activity area being within the display area.

[0021] Optionally, the method further comprises:

[0022] Determine a target area of ​​the sight line activity area outside the display area;

[0023] In response to the target area being greater than a preset threshold, determining that the sight line activity area is outside the display area;

[0024] In response to the target area being less than or equal to a preset threshold, it is determined that the sight line activity area is within the display area.

[0025] Optionally, after outputting the prompt information in response to the sight line activity area being outside the display area, the method further includes:

[0026] Determining a duration of time that the sight line activity area is outside the display area;

[0027] In response to the duration being greater than a preset duration, the screen is controlled to be in an off state.

[0028] In a second aspect, an embodiment of the present invention further provides a control device for a head mounted device, the control device for the head mounted device comprising:

[0029] A first determining module, configured to determine a visual activity area of ​​a user of the head mounted device in response to the head mounted device being in a wearing state;

[0030] A second determination module, used to determine a display area of ​​the head mounted device screen;

[0031] The output module is used for outputting prompt information in response to the visual line activity area being outside the display area, so as to prompt the user to adjust the wearing parameters of the head mounted device through the prompt information.

[0032] In a third aspect, an embodiment of the present invention further provides a head-mounted device, comprising a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the control method of the head-mounted device as described in the first aspect is implemented.

[0033] In a fourth aspect, an embodiment of the present invention further provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the control method of the head-mounted device as described in the first aspect.

[0034] The embodiment of the present invention provides a control method, device, electronic device and storage medium of a head-mounted device. The technical solution provided by the embodiment of the present invention realizes

[0035] The embodiment of the present application outputs a prompt message when the head-mounted device is in a wearing state, if the visual activity area of ​​the head-mounted device user is outside the display area of ​​the screen, so as to prompt the user to adjust the wearing parameters of the head-mounted device through the prompt message. This achieves the ability to timely detect the wearing posture of the user when wearing the head-mounted device. When it is determined that the visual activity area of ​​the user is outside the display area of ​​the screen, it is determined that the user's wearing posture is incorrect. At this time, a prompt message is output to prompt the user to adjust the wearing parameters of the head-mounted device, that is, to adjust the wearing posture, thereby avoiding the user from not being able to see the content displayed on the screen of the head-mounted device due to incorrect wearing posture, and also reducing the risk of the head-mounted device falling off when the user wears the head-mounted device, thereby improving the safety of the head-mounted device and enabling the user to wear the head-mounted device correctly. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0037] Figure 1 is a schematic structural block diagram of a head mounted device provided by an embodiment of the present invention;

[0038] Figure 2 is a flow chart of a control method of a head mounted device provided by an embodiment of the present invention;

[0039] Figure 3 is a flow chart of another method for controlling a head mounted device provided by an embodiment of the present invention;

[0040] Figure 4 The present invention is a schematic block diagram of a control device for a head mounted device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0043] It should be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0044] Data transmission between electronic devices can be achieved using transmission protocols such as Bluetooth, SPI (Serial Peripheral Interface), UART (Universal Asynchronous Receiver / Transmitter), and WIFI. Different transmission protocols have different corresponding bandwidths. For data transmission between different electronic devices, how to adapt to the bandwidth of different transmission protocols and achieve efficient data transmission is an urgent problem to be solved.

[0045] To solve the above problems, the embodiments of the present invention provide a control method, device, electronic device and storage medium for a head mounted device. The embodiments of the present invention provide a technical solution.

[0046] In the embodiment of the present invention, the head mounted device may be an AR (Augmented Reality) device, a VR (Virtual Reality) device, etc.

[0047] Some embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0048] Figure 1 The present invention is a schematic structural block diagram of a head mounted device provided by an embodiment of the present invention.

[0049] like Figure 1 As shown, the head mounted device 100 includes a processor 101 and a memory 102, and the processor 101 and the memory 102 are connected via a bus 103, such as an I2C (Inter-integrated Circuit) bus. The electronic device 100 can be a target electronic device, a first electronic device, or a second electronic device.

[0050] Specifically, the processor 101 is used to provide computing and control capabilities to support the operation of the entire electronic device 100. The processor 101 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0051] Specifically, the memory 102 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0052] Those skilled in the art will understand that Figure 1 The structure shown in the figure is only a block diagram of a partial structure related to the embodiment of the present invention, and does not constitute a limitation on the electronic device to which the embodiment of the present invention is applied. The specific electronic device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0053] The processor 101 is used to run a computer program stored in the memory 102. The computer program can be set in the target head-mounted device, and implement any one of the head-mounted device control methods provided by the embodiments of the present invention when the computer program is executed.

[0054] In one embodiment, the processor is used to run a computer program stored in the memory, and implements the following steps when executing the computer program:

[0055] In response to the head mounted device being in a wearing state, determining a visual activity area of ​​a user of the head mounted device;

[0056] Determining a display area of ​​the head mounted device screen;

[0057] In response to the sight line activity area being outside the display area, outputting prompt information to prompt the user to adjust the wearing parameters of the head mounted device through the prompt information

[0058] In one embodiment, in response to the sight line activity area being outside the display area, outputting prompt information to prompt the user to adjust the wearing parameters of the head mounted device through the prompt information, includes:

[0059] In response to the sight line activity area being outside the display area, determining a target number of times the sight line activity area is outside the display area within a first preset time period;

[0060] If the target number of times is greater than the preset number of times, a prompt message is output to prompt the user to adjust the wearing parameters of the head mounted device through the prompt message.

[0061] In one embodiment, in response to the head mounted device being in a wearing state, determining the visual activity area of ​​the head mounted device user includes:

[0062] In response to the head mounted device being in a wearing state, when it is detected that the eyeball position of the user of the head mounted device is in a changing state, determining an intersection point between the vertical line corresponding to the eyeball and the screen;

[0063] The intersection points corresponding to the second preset time length are obtained, and the visual line activity area of ​​the user is determined according to the intersection points corresponding to the second preset time length.

[0064] In one embodiment, the obtaining of the intersection points corresponding to the second preset time length, and determining the visual activity area of ​​the user according to the intersection points corresponding to the second preset time length, includes:

[0065] Acquire the intersection points corresponding to the second preset time length, and determine the intersection area of ​​the intersection points corresponding to the second preset time length, wherein all the intersection points corresponding to the second preset time length exist in the intersection area;

[0066] The intersection area of ​​the intersection points corresponding to the second preset time length is determined as the visual line activity area of ​​the user.

[0067] In one embodiment, the method comprises:

[0068] Establishing a coordinate system with the display area of ​​the head mounted device, determining vertex coordinates of four vertices corresponding to the display area in the coordinate system, and determining intersection coordinates of each intersection within the second preset time length in the coordinate system;

[0069] The relative position relationship between the sight line activity area and the display area is determined according to the vertex coordinates and the intersection coordinates, wherein the relative position relationship includes the sight line activity area being outside the display area and the sight line activity area being within the display area.

[0070] In one embodiment, the method further comprises:

[0071] Determine a target area of ​​the sight line activity area outside the display area;

[0072] In response to the target area being greater than a preset threshold, determining that the sight line activity area is outside the display area;

[0073] In response to the target area being less than or equal to a preset threshold, it is determined that the sight line activity area is within the display area.

[0074] In one embodiment, after outputting the prompt information in response to the sight line activity area being outside the display area, the method further includes:

[0075] Determining a duration of time that the sight line activity area is outside the display area;

[0076] In response to the duration being greater than a preset duration, the screen is controlled to be in an off state.

[0077] It should be noted that technicians in the relevant field can clearly understand that for the convenience and brevity of description, the specific working process of the head-mounted device described above can refer to the corresponding process in the control method embodiment of the head-mounted device described below, and will not be repeated here.

[0078] The following will be combined Figure 1 The electronic device in the embodiment of the present invention will introduce the control method of the head mounted device in detail. Figure 1 The head mounted device is only used to explain the control method of the head mounted device provided in the embodiment of the present invention, but does not constitute a limitation on the application scenario of the control method of the head mounted device provided in the embodiment of the present invention.

[0079] See also Figure 2 , Figure 2 It is a flowchart of a control method of a head mounted device provided by an embodiment of the present invention.

[0080] like Figure 2 As shown, the control method of the head mounted device is applied to the head mounted device, and the control method of the head mounted device includes steps S101 to S102.

[0081] Step S101: in response to the head mounted device being in a wearing state, determining a visual activity area of ​​a user of the head mounted device;

[0082] When the head mounted device is in a wearing state, the head mounted device determines the visual activity area of ​​its user. A sensor for detecting whether the head mounted device is in a wearing state may be provided in the head mounted device. For example, when the head mounted device is a smart glasses, the sensor may be provided in the frame at a position corresponding to the center of the user's eyebrows, and the sensor may be a proximity light sensor.

[0083] In one embodiment, step S101 includes: in response to the head mounted device being in a wearing state, when it is detected that the eyeball position of the head mounted device user is in a changing state, determining an intersection point between the vertical line corresponding to the eyeball and the screen;

[0084] The intersection points corresponding to the second preset time length are obtained, and the visual line activity area of ​​the user is determined according to the intersection points corresponding to the second preset time length.

[0085] In one embodiment, when the head mounted device is in a wearing state, the head mounted device detects whether the eye position of its user is in a changing state. For example, the head mounted device can detect whether the eye position of the user is in a changing state by using eye tracking technology. For example, when the coordinate position of the user's eye on the screen of the head mounted device changes within a set time, it can be determined that the user's eye position is in a changing state; if the coordinate position of the user's eye on the screen of the head mounted device does not change within the set time, it can be determined that the user's eye position is not in a changing state. It can be understood that in one embodiment, when the coordinates of the user's eye on the screen at a first time point are different from the coordinates on the screen at a second time point, it can be considered that the eye position is in a changing state, wherein the first time point and the second time point are time points within the set time. In another embodiment, when the distance between the coordinates of the user's eye on the screen at a first time point and the coordinates on the screen at a second time point is greater than a preset value, it is determined that the eye position is in a changing state, and the preset value can be set according to specific needs, and this embodiment does not limit this.

[0086] When the head-mounted device is in a wearing state, and it is detected that the eye position of the head-mounted device user is in a changing state, the intersection between the vertical line corresponding to the eye and the screen is determined. Among them, the vertical line corresponding to the eye includes a left vertical line corresponding to the left eye and a right vertical line corresponding to the right eye. The vertical line is a line perpendicular to the screen, that is, the vertical line is a line perpendicular to the plane where the display area of ​​the head-mounted device is located. For example, when the head-mounted device is a smart glasses, the vertical line is a line perpendicular to the frame of the smart glasses. The screen of the smart glasses is a virtual screen for displaying information, and the area of ​​the display area of ​​the smart glasses is less than or equal to the area of ​​its screen. The head-mounted device obtains the changes of the two eyeballs of the user in the second preset time length, determines the intersection between the vertical line corresponding to the left eye and the screen within the second preset time length, and determines the intersection between the vertical line corresponding to the right eye and the screen within the second preset time length. In this embodiment, for the sake of easy distinction, the intersection point corresponding to the left eye is recorded as the first intersection point, and the intersection point corresponding to the right eye is recorded as the second intersection point. It can be understood that within the second preset time length, there will be multiple first intersection points and multiple second intersection points, and there are at least two intersection points with different coordinates among the multiple first intersection points, and there are at least two intersection points with different coordinates among the multiple second intersection points. The second preset time length can be set according to specific needs, and this embodiment does not limit this. When the head-mounted device obtains multiple first intersection points and multiple second intersection points within the second preset time length, the user's line of sight activity area is determined based on the multiple first intersection points and the multiple second intersection points.

[0087] In one embodiment, the obtaining of the intersection points corresponding to the second preset time length, and determining the visual activity area of ​​the user according to the intersection points corresponding to the second preset time length, includes:

[0088] Acquire the intersection points corresponding to the second preset time length, and determine the intersection area of ​​the intersection points corresponding to the second preset time length, wherein all the intersection points corresponding to the second preset time length exist in the intersection area;

[0089] The intersection area of ​​the intersection points corresponding to the second preset time length is determined as the visual line activity area of ​​the user.

[0090] When the head-mounted device obtains the corresponding intersection within the second preset time length, the intersection area corresponding to the intersection is determined, and the intersection area corresponding to the intersection is determined as the user's line of sight activity area. Among them, the intersection area determined by the left eye corresponding intersection within the second preset time length is the left eye intersection area, and all left eye corresponding intersections obtained within the second preset time length exist in the left eye intersection area. The intersection area determined by the right eye corresponding intersection within the second preset time length is the right eye intersection area, and all right eye corresponding intersections obtained within the second preset time length exist in the right eye intersection area. The line of sight activity areas mentioned in the embodiments of the present application are the line of sight activity areas of the user's right eye and the line of sight activity areas of the user's left eye.

[0091] For example, a circle or a rectangular frame can be drawn to surround all the first intersections, and the intersection area corresponding to the circle or rectangular frame is the visual activity area corresponding to the user's left eye, that is, the left eye intersection area; a circle or a rectangular frame can be drawn to surround all the second intersections, and the intersection area corresponding to the circle or rectangular frame is the visual activity area corresponding to the user's right eye, that is, the right eye intersection area. It can be understood that in some embodiments, the visual activity area of ​​the user's left eye and the visual activity area of ​​the user's right eye may have a partially overlapping area, or there may be no overlapping area.

[0092] In another embodiment, in order to ensure that the user's visual activity area can be accurately determined, the head mounted device can determine the user's left eye intersection area based on the first intersection when the number of first intersections collected is greater than or equal to a preset number, and continue to collect the first intersection when the number of first intersections is less than the preset number; determine the user's right eye intersection area based on the second intersection when the number of second intersections collected is greater than or equal to the preset number, and continue to collect the second intersection when the number of second intersections is less than the preset number. The preset number can be set to 30, 50 or 80, etc.

[0093] Step S102: Determine the display area of ​​the head mounted device screen.

[0094] The display area of ​​the head-mounted device screen can be pre-stored in the head-mounted device, such as the size of the display area can be pre-stored in the head-mounted device, that is, the display parameters corresponding to the display area can be pre-stored. For example, when the head-mounted device is a smart glasses, the screen of the binocular smart glasses will have two display areas, namely, a first display area corresponding to the left lens, and a second display area corresponding to the right lens, and the display area of ​​each lens is also pre-set. In one embodiment, the size of the first display area and the second display area is the same; in another embodiment, the size of the first display area and the second display area may also be different.

[0095] If there is only one display area on the screen of the head mounted device, in the process of determining the display area, the center point of the display area can be first determined. The center point can be the position where the user's eyes are looking forward when the head mounted device is in the wearing state, that is, the position where the user's eyes are looking forward at this time is determined as the center point of the display area, and then the display parameters corresponding to the display area are obtained, and the display area is determined according to the display parameters and the center point. In one embodiment, the display parameter can be a coordinate point in a plane coordinate system established with the center point as the coordinate origin, the right direction as the positive direction of the X axis, and the top as the positive direction of the Y axis. At this time, there are at least 2 coordinate points in the display parameter. When there are 2 coordinate points in the display parameter, these 2 coordinate points are diagonal coordinate points. In another embodiment, the display parameter can be the distance between the corner point of the display area and the coordinate origin.

[0096] Step S103: In response to the sight line activity area being outside the display area, outputting prompt information to prompt the user to adjust wearing parameters of the head mounted device through the prompt information.

[0097] After determining the display area of ​​the head-mounted device screen, the head-mounted device determines whether the user's line of sight activity area is outside the display area. If it is determined that the user's line of sight activity area is outside the display area, the head-mounted device outputs a prompt message to prompt the user to adjust its wearing parameters through the prompt message. The head-mounted device can output the prompt message through voice, text, or indicator light. The wearing parameter can be a wearing position, such as prompting the user to adjust the worn head-mounted device upwards through a prompt message, or prompting the user to adjust the worn head-mounted device downwards through a prompt message. In another embodiment, if the user's line of sight activity area is within the display area, the head-mounted device does not output a prompt message to maintain the current state.

[0098] In one embodiment, if the user's sight line activity area has no intersection with the display area, it is determined that the user's sight line activity area is outside the display area. For example, when the head-mounted device is a smart glasses, if the user's left eye sight line activity area has no intersection with the first display area, and the user's right eye sight line activity area has no intersection with the second display area, it is determined that the user's sight line activity area is outside the display area of ​​the screen; if the user's left eye sight line activity area has an intersection with the first display area, and / or the user's right eye sight line activity area has an intersection with the second sight line display area, it is determined that the user's sight line activity area is within the display area of ​​the screen.

[0099] In another embodiment, if the user's visual activity area and the display area have an intersection, but the ratio of the area of ​​the intersection of the user's visual activity area and the display area to the display area of ​​the screen is less than a preset value, it is determined that the user's visual activity area is outside the display area of ​​the screen, and the preset value can be set according to specific needs, such as 10%, 15%, etc. For example, when the head-mounted device is a smart glasses, the first area of ​​the intersection between the visual activity area of ​​the user's left eye and the first display area can be determined, and the second area of ​​the intersection between the visual activity area of ​​the user's right eye and the second display area can be determined, and the sum of the first area and the second area can be calculated to obtain a third area, and the third area is divided by the total area of ​​the display area of ​​the screen to obtain the ratio of the area of ​​the intersection of the user's visual activity area and the display area to the display area of ​​the screen. The total area of ​​the display area of ​​the screen is equal to the sum of the area of ​​the first display area and the area of ​​the second display area. Among them, the first area and the second area can be equal or unequal. In this embodiment, the ratio is expressed as a percentage, and in other embodiments, the ratio can also be expressed in other forms. When the ratio is less than a preset value, it is determined that the user's visual activity area is outside the display area of ​​the screen; if the ratio is greater than or equal to the preset value, it is determined that the user's visual activity area is within the display area of ​​the screen.

[0100] This embodiment outputs a prompt message when the head-mounted device is in a wearing state, if the visual activity area of ​​the head-mounted device user is outside the display area of ​​the screen, so as to prompt the user to adjust the wearing parameters of the head-mounted device through the prompt message. This realizes timely detection of the wearing posture of the user when wearing the head-mounted device. When it is determined that the visual activity area of ​​the user is outside the display area of ​​the screen, it is determined that the user's wearing posture is incorrect. At this time, a prompt message is output to prompt the user to adjust the wearing parameters of the head-mounted device, that is, to adjust the wearing posture, thereby avoiding the user from not being able to see the content displayed on the screen of the head-mounted device due to incorrect wearing posture, and also reducing the risk of the head-mounted device falling off when the user wears the head-mounted device, thereby improving the safety of the head-mounted device and enabling the user to wear the head-mounted device correctly.

[0101] In one embodiment, step S103 includes: sub-step a and sub-step b.

[0102] Sub-step a: in response to the sight line activity area being outside the display area, determining a target number of times the sight line activity area is outside the display area within a first preset time period.

[0103] Sub-step b: if the target number of times is greater than a preset number of times, outputting a prompt message to prompt the user to adjust the wearing parameters of the head mounted device through the prompt message.

[0104] When it is determined that the user's line of sight activity area is outside the display area of ​​the head-mounted device screen, the head-mounted device determines the target number of times the line of sight activity area is outside the display area within the first preset duration. It is understandable that the head-mounted device can detect whether the user's line of sight activity area is outside the display area at a preset frequency. The preset frequency and the first preset duration can be set according to specific needs. This embodiment does not impose specific restrictions on this, but the setting of the preset frequency and the first preset duration must ensure that within the first preset duration, the head-mounted device can obtain the user's line of sight activity area N times to perform N judgment operations, which are used to determine whether the user's line of sight activity area is outside the display area. Wherein, N is a positive integer greater than 0.

[0105] The target number is the number of times the user's line of sight activity area is outside the display area within the first preset time length. After the head-mounted device obtains the target number, the head-mounted device determines whether the target number is greater than the preset number. Among them, the preset number is less than N, and this embodiment does not specifically limit the size of the preset number. If it is determined that the target number is greater than the preset number, the head-mounted device outputs a prompt message to prompt the user to adjust the wearing parameters of the head-mounted device through the prompt message. In another embodiment, if it is determined that the target number is less than or equal to the preset number, the head-mounted device does not output a prompt message, and re-determines whether the target number of times the line of sight activity area is outside the screen display area within the first preset time length is greater than the preset number.

[0106] This embodiment determines that the user's posture in wearing the head-mounted device is incorrect if the target number of times the user's line of sight activity area is outside the screen display area is greater than a preset number within a first preset time period, and outputs a prompt message to prompt the user to adjust the wearing parameters of the head-mounted device to improve the accuracy of detecting the wearing posture of the user wearing the head-mounted device.

[0107] In one embodiment, the method comprises:

[0108] Establishing a coordinate system with the display area of ​​the head mounted device, determining vertex coordinates of four vertices corresponding to the display area in the coordinate system, and determining intersection coordinates of each intersection within the second preset time length in the coordinate system;

[0109] The relative position relationship between the sight line activity area and the display area is determined according to the vertex coordinates and the intersection coordinates, wherein the relative position relationship includes the sight line activity area being outside the display area and the sight line activity area being within the display area.

[0110] In one embodiment, a plane coordinate system is established with the display area of ​​the head-mounted device. The coordinate system can also use one of the points in the display area as the coordinate origin, such as using the lower left corner of the display area as the coordinate origin, the long direction of the display area (right direction) as the positive direction of the X axis of the plane coordinate system, and the wide direction of the display area (upward direction) as the positive direction of the Y axis of the plane coordinate system. It can be understood that the above is only an example of establishing a coordinate system, and the coordinate system can also be established in other ways. After the coordinate system is established, the head-mounted device determines the vertex coordinates of the four vertices corresponding to its display area in the coordinate system, that is, the area surrounded by these four vertex coordinates is the display area. For example, the four vertex coordinates corresponding to the display area are the first coordinate of the lower left corner of the display area, the second coordinate of the upper left corner of the display area, the third coordinate of the lower right corner of the display area, and the fourth coordinate of the upper right corner of the display area. It can be understood that when the head-mounted device is an AR glasses, the display area is the area where the AR glasses display content.

[0111] The head mounted device can determine the intersection coordinates of each intersection in the coordinate system according to the position of each intersection in the display area within the second preset time period. After the vertex coordinates and the intersection coordinates are determined, the head mounted device can determine the relative position relationship between the sight line activity area and the display area according to the vertex coordinates and the intersection coordinates. The relative position relationship includes the sight line activity area being outside the display area and the sight line activity area being within the display area, that is, whether the sight line activity area is within the display area can be determined according to the relative position relationship.

[0112] There are four situations in which the intersection point is outside the display area within the second preset duration:

[0113] ① The intersection point corresponding to the intersection coordinates of the intersection point whose abscissa is smaller than the abscissa of the first coordinate is outside the display area;

[0114] ② The intersection point corresponding to the intersection point whose ordinate is smaller than the ordinate of the first coordinate is outside the display area;

[0115] ③ The intersection point corresponding to the intersection coordinate of the ordinate greater than the ordinate of the second coordinate is outside the display area;

[0116] ④ The intersection point corresponding to the intersection coordinates of the abscissa greater than the abscissa of the third coordinate is outside the display area.

[0117] After determining whether the intersection is outside the display area within the second preset time, the head mounted device can be based on the relative position relationship between the sight line activity area and the display area. For example, when all the intersection coordinates within the second preset time are within the display area, the relative position relationship determined is that the sight line activity area is within the display area; when there is an intersection outside the display area within the second preset time, the relative position relationship determined is that the sight line activity area is outside the display area.

[0118] It can be understood that this embodiment is applicable to the case where the head-mounted device has one display area and multiple display areas. When the head-mounted device is an AR glasses, each lens of the AR glasses can correspond to a display area. At this time, the scheme of this embodiment can be used to determine the relative position relationship between the left eye display area and the left eye sight activity area, and the scheme of this embodiment can be used to determine the relative position relationship between the right eye display area and the right eye sight activity area; when the AR glasses correspond to only one display area, the relative position relationship between the display area and the total sight activity area can also be determined by the scheme of this embodiment. The total sight activity is determined by the user's left eye display area and right eye display area.

[0119] This embodiment determines the relative position relationship between the line of sight activity area and the display area based on the positional relationship between the vertex coordinates of the display area and the corresponding intersection coordinates of the line of sight activity area, that is, determines the relative position relationship between the line of sight activity area and the display area through simple mathematical calculations, thereby improving the confirmation rate of the relative position relationship between the line of sight activity area and the display area.

[0120] In another embodiment, the method further comprises:

[0121] Determine a target area of ​​the sight line activity area outside the display area;

[0122] In response to the target area being greater than a preset threshold, determining that the sight line activity area is outside the display area;

[0123] In response to the target area being less than or equal to a preset threshold, it is determined that the sight line activity area is within the display area.

[0124] When the head-mounted device determines that part of the line of sight activity area is outside the display area (that is, at this time, at least one of the intersections within the second preset time length is outside the display area), the head-mounted device determines the target area of ​​the line of sight activity area outside the display area, and determines whether the target area is greater than the preset threshold. If the target area is greater than the preset threshold, the head-mounted device determines that the line of sight activity area is outside the display area; if the target area is less than or equal to the preset threshold, the head-mounted device determines that the line of sight activity area is within the display area. Among them, the preset threshold can be set according to specific needs, and this embodiment does not specifically limit the size of the preset threshold.

[0125] This embodiment determines whether the line of sight activity area is within the display area through the area relationship between the two areas. When the target area of ​​the line of sight activity area outside the display area is smaller than a preset threshold, it indicates that most of the area of ​​the line of sight activity area is still within the display area, that is, the user can still view most of the content displayed in the display area. This embodiment avoids outputting prompt information when the user can still view most of the content in the display area, thereby reducing the frequency of outputting prompt information and improving the accuracy of outputting prompt information.

[0126] Reference Figure 3 In another embodiment of the present application, the control method of the head mounted device further includes:

[0127] Step S104: determining a duration in which the sight line activity area is outside the display area.

[0128] Step S105: In response to the duration being greater than a preset duration, the screen is controlled to be in an off state.

[0129] When it is determined that the sight line activity area is outside the display area, the head-mounted device determines the duration of the sight line activity area outside the display area, and determines whether the duration is greater than the preset duration; if it is determined that the sight line activity area is outside the display area for a duration greater than the preset duration, the head-mounted device controls its screen to be in an off state; if it is determined that the sight line activity area is outside the display area for a duration less than or equal to the preset duration, the head-mounted device controls its screen to maintain the current state. Among them, this embodiment does not specifically limit the size of the preset duration, such as the preset duration can be set to 5 minutes, 10 minutes or 3 minutes, etc.

[0130] This embodiment saves power consumption of the head-mounted device by controlling the screen to be in an off state when the visual activity area is outside the display area for a duration greater than a preset time, thereby improving the battery life of the head-mounted device.

[0131] See also Figure 4 , Figure 4 It is a schematic block diagram of the structure of a control device for a head mounted device provided in an embodiment of the present invention.

[0132] like Figure 4 As shown, the control device 200 of the head mounted device is applied to the head mounted device, and the control device 200 of the head mounted device includes:

[0133] A first determining module 210, configured to determine a visual activity area of ​​a user of the head mounted device in response to the head mounted device being in a wearing state;

[0134] A second determination module 220, configured to determine a display area of ​​the head mounted device screen;

[0135] The output module 230 is used to output prompt information in response to the visual activity area being outside the display area, so as to prompt the user to adjust the wearing parameters of the head mounted device through the prompt information.

[0136] In some embodiments, the output module 230 includes:

[0137] A times determining unit, configured to determine a target times of the sight line activity area being outside the display area within a first preset time period in response to the sight line activity area being outside the display area;

[0138] An information output unit is used to output a prompt message if the target number of times is greater than a preset number of times, so as to prompt the user to adjust the wearing parameters of the head mounted device through the prompt message.

[0139] In some embodiments, the first determining module 210 further includes:

[0140] an intersection determination unit, configured to determine an intersection point between a vertical line corresponding to the eyeball and the screen in response to the head mounted device being in a wearing state and when detecting that the eyeball position of the user of the head mounted device is in a changing state;

[0141] The area determination unit is used to obtain the intersection points corresponding to the second preset time length, and determine the user's line of sight activity area according to the intersection points corresponding to the second preset time length.

[0142] In some embodiments, the area determination unit is also used to obtain the intersection points corresponding to a second preset time length, and determine the intersection area of ​​the intersection points corresponding to the second preset time length, wherein all intersection points corresponding to the second preset time length exist in the intersection area; and the intersection area of ​​the intersection points corresponding to the second preset time length is determined as the user's line of sight activity area.

[0143] In some embodiments, the control device of the head mounted device further includes:

[0144] a coordinate determination module, configured to establish a coordinate system using the display area of ​​the head mounted device, determine vertex coordinates of four vertices corresponding to the display area in the coordinate system, and determine intersection coordinates of each intersection within the second preset time period in the coordinate system;

[0145] A position relationship determination module is used to determine the relative position relationship between the line of sight activity area and the display area according to the vertex coordinates and the intersection coordinates, wherein the relative position relationship includes the line of sight activity area outside the display area and the line of sight activity area within the display area.

[0146] In some embodiments, the control device of the head mounted device further includes:

[0147] An area determination module, used for determining a target area of ​​the sight line activity area outside the display area;

[0148] The third determination module is used to determine that the line of sight activity area is outside the display area in response to the target area being larger than a preset threshold; and to determine that the line of sight activity area is within the display area in response to the target area being smaller than or equal to the preset threshold.

[0149] In some embodiments, the control device of the head mounted device further includes:

[0150] A duration determination module, used to determine the duration of the sight line activity area being outside the display area;

[0151] The screen-off control module is used to control the screen to be in an off state in response to the duration being greater than a preset duration.

[0152] It should be noted that, those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the control device of the head-mounted device described above can refer to the corresponding process in the aforementioned head-mounted device control method embodiment, and will not be repeated here.

[0153] An embodiment of the present invention also provides a storage medium for computer-readable storage, wherein the storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement any method for controlling a head-mounted device provided in the description of the embodiment of the present invention.

[0154] The storage medium may be an internal storage unit of the electronic device described in the above embodiment, such as a hard disk or memory of the electronic device. The storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device.

[0155] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware embodiment, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically contain computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0156] It should be understood that the term "and / or" used in the present specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, including these combinations. It should be noted that, in this article, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a..." does not exclude the presence of other identical elements in the process, method, article or system including the element.

[0157] The serial numbers of the embodiments of the present invention are only for description and do not represent the advantages and disadvantages of the embodiments. The above description is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A method for controlling a head mounted device, characterized in that: The method comprises: In response to the head mounted device being in a wearing state, determining a visual activity area of ​​a user of the head mounted device; Determining a display area of ​​the head mounted device screen; In response to the sight line activity area being outside the display area, prompt information is output to prompt the user to adjust the wearing parameters of the head mounted device through the prompt information.

2. The method for controlling a head mounted device according to claim 1, wherein: The step of outputting prompt information in response to the sight line activity area being outside the display area, so as to prompt the user to adjust the wearing parameters of the head mounted device through the prompt information, comprises: In response to the sight line activity area being outside the display area, determining a target number of times the sight line activity area is outside the display area within a first preset time period; If the target number of times is greater than the preset number of times, a prompt message is output to prompt the user to adjust the wearing parameters of the head mounted device through the prompt message.

3. The control method of the head mounted device according to claim 1, characterized in that: In response to the head mounted device being in a wearing state, determining a visual activity area of ​​a user of the head mounted device includes: In response to the head mounted device being in a wearing state, when it is detected that the eyeball position of the user of the head mounted device is in a changing state, determining an intersection point between the vertical line corresponding to the eyeball and the screen; The intersection points corresponding to the second preset time length are obtained, and the visual line activity area of ​​the user is determined according to the intersection points corresponding to the second preset time length.

4. The method for controlling a head mounted device according to claim 3, wherein: The obtaining of the intersection point corresponding to the second preset time length, and determining the visual activity area of ​​the user according to the intersection point corresponding to the second preset time length, includes: Acquire the intersection points corresponding to the second preset time length, and determine the intersection area of ​​the intersection points corresponding to the second preset time length, wherein all the intersection points corresponding to the second preset time length exist in the intersection area; The intersection area of ​​the intersection points corresponding to the second preset time length is determined as the visual line activity area of ​​the user.

5. The method for controlling a head mounted device according to claim 3, wherein: The method comprises: Establishing a coordinate system with the display area of ​​the head mounted device, determining vertex coordinates of four vertices corresponding to the display area in the coordinate system, and determining intersection coordinates of each intersection within the second preset time length in the coordinate system; The relative position relationship between the sight line activity area and the display area is determined according to the vertex coordinates and the intersection coordinates, wherein the relative position relationship includes the sight line activity area being outside the display area and the sight line activity area being within the display area.

6. The method for controlling a head mounted device according to claim 1, wherein: The method further comprises: Determine a target area of ​​the sight line activity area outside the display area; In response to the target area being greater than a preset threshold, determining that the sight line activity area is outside the display area; In response to the target area being less than or equal to a preset threshold, it is determined that the sight line activity area is within the display area.

7. The method for controlling a head mounted device according to any one of claims 1 to 6, characterized in that: After the prompt information is output in response to the sight line activity area being outside the display area, the method further includes: Determining a duration of time that the sight line activity area is outside the display area; In response to the duration being greater than a preset duration, the screen is controlled to be in an off state.

8. A control device for a head mounted device, characterized in that: The control device of the head-mounted device is applied to the head-mounted device, and the control device of the head-mounted device includes: A first determining module, configured to determine a visual activity area of ​​a user of the head mounted device in response to the head mounted device being in a wearing state; A second determination module, used to determine a display area of ​​the head mounted device screen; The output module is used for outputting prompt information in response to the visual line activity area being outside the display area, so as to prompt the user to adjust the wearing parameters of the head mounted device through the prompt information.

9. A head mounted device, characterized in that: The head-mounted device includes a processor, a memory, a computer program stored in the memory and executable by the processor, and a data bus for realizing connection and communication between the processor and the memory, wherein when the computer program is executed by the processor, the control method of the head-mounted device as described in any one of claims 1 to 7 is implemented.

10. A storage medium for computer-readable storage, characterized in that: The storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the control method of the head mounted device according to any one of claims 1 to 7.

Images