Head-mounted device interaction control method and device, head-mounted device and storage medium

Abstract

The invention provides a head-mounted device interaction control method and device, a head-mounted device and a computer readable storage medium. The head-mounted device interaction control method comprises the following steps: when a to-be-processed incoming call exists in the head-mounted device, acquiring sensor measurement data of the head-mounted device; performing identification processing based on the sensor measurement data to obtain a relative motion orientation of the head-mounted device; determining a wearer response action of the head-mounted device based on the relative motion orientation; and triggering a target processing instruction corresponding to the response action of the wearer so as to process the incoming call to be processed through the target processing instruction. According to the invention, the incoming call reminding can be processed when the two hands of the user cannot be liberated, the processing convenience of the incoming call reminding of the head-mounted device is improved, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the technical field of head-mounted devices, and in particular to a head-mounted device interactive control method, device, head-mounted device, and computer-readable storage medium. Background Art

[0002] With the development of technology, head-mounted devices such as AR (Augmented Reality) glasses and VR (Virtual Reality) glasses are becoming more and more popular among users. In the related art, physical controls such as sliders and buttons are set on the head-mounted device or on the interactive control device of the head-mounted device to process the incoming call reminder; however, when the user's hands cannot be freed, the incoming call reminder cannot be processed. Summary of the invention

[0003] The present application provides a head-mounted device interactive control method, device, head-mounted device and computer-readable storage medium, which can realize the processing of incoming call reminders when the user's hands cannot be freed, improve the convenience of processing incoming call reminders of the head-mounted device, and thus improve the user experience.

[0004] In a first aspect, the present application provides a head mounted device interactive control method, the method comprising:

[0005] When the head mounted device has an incoming call to be processed, obtaining sensor measurement data of the head mounted device;

[0006] Performing identification processing based on the sensor measurement data to obtain the relative motion orientation of the head mounted device;

[0007] Determining a response action of a wearer of the head mounted device based on the relative motion orientation;

[0008] A target processing instruction corresponding to the wearer's response action is triggered to process the incoming call to be processed through the target processing instruction.

[0009] In some embodiments, determining a response action of a wearer of the head mounted device based on the relative motion orientation includes:

[0010] If the relative movement direction is returning to the original position after shaking the head to the left and / or returning to the original position after shaking the head to the right, the shaking head action is taken as the response action of the wearer of the head mounted device;

[0011] Alternatively, if the relative movement direction is returning to the original position after shaking the head upward and / or returning to the original position after shaking the head downward, a nodding action is used as a response action of the wearer of the head-mounted device.

[0012] In some embodiments, triggering a target processing instruction corresponding to the wearer's response action to process the pending incoming call through the target processing instruction includes:

[0013] When the wearer's response action is a head shaking action, triggering a rejection processing instruction corresponding to the head shaking action, so as to process the pending incoming call through the rejection processing instruction;

[0014] Alternatively, when the wearer's response action is a head shaking action, a connection processing instruction corresponding to the head shaking action is triggered, so as to process the incoming call to be processed through the connection processing instruction.

[0015] In some embodiments, triggering a target processing instruction corresponding to the wearer's response action to process the pending incoming call through the target processing instruction includes:

[0016] When the wearer's response action is a nod action, triggering a connection processing instruction corresponding to the nod action, so as to process the incoming call to be processed through the connection processing instruction;

[0017] Alternatively, when the wearer's response action is a nod action, a rejection processing instruction corresponding to the nod action is triggered, so as to process the pending incoming call through the rejection processing instruction.

[0018] In some embodiments, the sensor measurement data includes inertial measurement data, the inertial measurement data includes three-axis angular velocity data and three-axis acceleration data of the head-mounted device, and the identification processing based on the sensor measurement data to obtain the relative motion orientation of the head-mounted device includes:

[0019] Based on the three-axis angular velocity data and the three-axis acceleration data, obtaining the movement direction angle of the head mounted device at each time point;

[0020] Acquire the time point when the motion direction angle is greater than a preset angle threshold as the turning point time point;

[0021] Determining a reset time point of the head mounted device based on the three-axis angular velocity data and the three-axis acceleration data;

[0022] The relative orientation of the device between the inflection point time point and the reset time point is obtained as the relative movement orientation of the head mounted device.

[0023] In some embodiments, the sensor measurement data further includes ranging sensor data, and obtaining the relative orientation of the device between the inflection point time point and the reset time point as the relative motion orientation of the head mounted device includes:

[0024] Determine the ground distance change between the inflection point time point and the reset time point according to the distance measurement sensor data;

[0025] The ground distance change is obtained, and the relative orientation of the device between the inflection point time point and the reset time point is determined as the relative movement orientation of the head mounted device.

[0026] In some embodiments, the method further comprises:

[0027] When the incoming call to be processed is connected, detecting the number of times the head-mounted device continues to shake its head;

[0028] If the head-mounted device continues shaking its head for more than a preset shaking head threshold, a hang-up processing instruction is executed to hang up the connected incoming call to be processed.

[0029] In a second aspect, the present application provides a head mounted device interactive control apparatus, the head mounted device interactive control apparatus comprising:

[0030] An acquisition unit, configured to acquire sensor measurement data of the head mounted device when there is an incoming call to be processed on the head mounted device;

[0031] An identification unit, configured to perform identification processing based on the sensor measurement data to obtain a relative motion orientation of the head mounted device;

[0032] a determining unit, configured to determine a response action of a wearer of the head mounted device based on the relative motion orientation;

[0033] A processing unit is used to trigger a target processing instruction corresponding to the wearer's response action, so as to process the incoming call to be processed through the target processing instruction.

[0034] In some embodiments, the determining unit is specifically configured to:

[0035] If the relative movement direction is returning to the original position after shaking the head to the left and / or returning to the original position after shaking the head to the right, the shaking head action is taken as the response action of the wearer of the head mounted device;

[0036] Alternatively, if the relative movement direction is returning to the original position after shaking the head upward and / or returning to the original position after shaking the head downward, a nodding action is used as a response action of the wearer of the head-mounted device.

[0037] In some embodiments, the processing unit is specifically configured to:

[0038] When the wearer's response action is a head shaking action, triggering a rejection processing instruction corresponding to the head shaking action, so as to process the pending incoming call through the rejection processing instruction;

[0039] Alternatively, when the wearer's response action is a head shaking action, a connection processing instruction corresponding to the head shaking action is triggered, so as to process the incoming call to be processed through the connection processing instruction.

[0040] In some embodiments, the processing unit is specifically configured to:

[0041] When the wearer's response action is a nod action, triggering a connection processing instruction corresponding to the nod action, so as to process the incoming call to be processed through the connection processing instruction;

[0042] Alternatively, when the wearer's response action is a nod action, a rejection processing instruction corresponding to the nod action is triggered, so as to process the pending incoming call through the rejection processing instruction.

[0043] In some embodiments, the sensor measurement data includes inertial measurement data, the inertial measurement data includes three-axis angular velocity data and three-axis acceleration data of the head mounted device, and the identification unit is specifically used to:

[0044] Based on the three-axis angular velocity data and the three-axis acceleration data, obtaining the movement direction angle of the head mounted device at each time point;

[0045] Acquire the time point when the motion direction angle is greater than a preset angle threshold as the turning point time point;

[0046] Determining a reset time point of the head mounted device based on the three-axis angular velocity data and the three-axis acceleration data;

[0047] The relative orientation of the device between the inflection point time point and the reset time point is obtained as the relative movement orientation of the head mounted device.

[0048] In some embodiments, the sensor measurement data further includes ranging sensor data, and the identification unit is specifically configured to:

[0049] Determine the ground distance change between the inflection point time point and the reset time point according to the distance measurement sensor data;

[0050] The ground distance change is obtained, and the relative orientation of the device between the inflection point time point and the reset time point is determined as the relative movement orientation of the head mounted device.

[0051] In some embodiments, the processing unit is specifically configured to:

[0052] When the incoming call to be processed is connected, detecting the number of times the head-mounted device continues to shake its head;

[0053] If the head-mounted device continues shaking its head for more than a preset shaking head threshold, a hang-up processing instruction is executed to hang up the connected incoming call to be processed.

[0054] In a third aspect, the present application also provides a head-mounted device, comprising a processor and a memory, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, it executes any one of the head-mounted device interactive control methods provided in the present application.

[0055] In a fourth aspect, the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program is loaded by a processor to execute the head-mounted device interactive control method.

[0056] The present application obtains inertial measurement data of the head-mounted device for identification and processing when there is a pending incoming call on the head-mounted device, thereby obtaining the relative movement orientation of the head-mounted device, which is used to determine the response action of the wearer of the head-mounted device, trigger the target processing instruction corresponding to the wearer's response action, and process the pending incoming call through the target processing instruction; thereby, the user can process the incoming call reminder of the head-mounted device through head movement, and the incoming call reminder can be processed when the user's hands cannot be freed, thereby improving the convenience of processing the incoming call reminder of the head-mounted device, and thus improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0058] Figure 1 is a schematic structural block diagram of a head mounted device provided in an embodiment of the present application;

[0059] Figure 2 It is a flow chart of a head mounted device interactive control method provided in an embodiment of the present application;

[0060] Figure 3 This is a flowchart of an embodiment of step 202 in the embodiment of the present application;

[0061] Figure 4 is a schematic diagram for explaining the change in ground distance in an embodiment of the present application;

[0062] Figure 5 It is a schematic diagram of the structure of an embodiment of the head-mounted device interactive control device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0063] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0064] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0065] In the description of the embodiments of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, the meaning of "multiple" is two or more, unless otherwise clearly and specifically defined.

[0066] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art can recognize that the present application can also be implemented without using these specific details. In other examples, the known process will not be elaborated in detail to avoid unnecessary details that make the description of the present application embodiment obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in accordance with the embodiments of the present application.

[0067] The embodiments of the present application provide a head-mounted device interactive control method, device, head-mounted device and computer-readable storage medium. Among them, the head-mounted device interactive control device can be integrated in the head-mounted device. Among them, the head-mounted device can be smart glasses, smart helmets, etc., smart glasses can be AR (augmented reality) glasses, VR (Virtual Reality) glasses, MR (Mixed Reality) glasses, XR (eXtended Reality) glasses, etc., and the smart helmet can be an AR helmet, etc.

[0068] The executor of the head-mounted device interactive control method of the embodiment of the present application may be the head-mounted device interactive control device provided in the embodiment of the present application, or a head-mounted device in which the head-mounted device interactive control device is integrated, wherein the head-mounted device interactive control device may be implemented in hardware or software.

[0069] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0070] Figure 1 It is a structural schematic block diagram of a head-mounted device provided in an embodiment of the present application.

[0071] like Figure 1 As shown, the head mounted device 100 includes a processor 101 and a memory 102, and the processor 101 and the memory 102 are connected via a bus 103, such as an I2C (Inter-integrated Circuit) bus.

[0072] Specifically, the processor 101 is used to provide computing and control capabilities to support the operation of the entire head-mounted device 100. The processor 101 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0073] Specifically, the memory 102 may be a Flash chip, a read-only memory (ROM) disk, an optical disk, a USB flash drive, or a mobile hard disk.

[0074] Those skilled in the art will understand that Figure 1 The structure shown in the figure is merely a block diagram of a partial structure related to the embodiment of the present application, and does not constitute a limitation on the head-mounted device to which the embodiment of the present application is applied. The specific head-mounted device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0075] The processor 101 is used to run the computer program stored in the memory 102, and implement any one of the head mounted device interactive control methods provided in the embodiments of the present application when executing the computer program. For example, the processor 101 is used to run the computer program stored in the memory 102, and can implement the following steps when executing the computer program:

[0076] When a head-mounted device has a pending incoming call, sensor measurement data of the head-mounted device is obtained; identification processing is performed based on the sensor measurement data to obtain the relative movement orientation of the head-mounted device; based on the relative movement orientation, a response action of the wearer of the head-mounted device is determined; and a target processing instruction corresponding to the wearer's response action is triggered to process the pending incoming call through the target processing instruction.

[0077] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0078] If the relative movement direction is returning to the original position after shaking the head to the left and / or returning to the original position after shaking the head to the right, the shaking head action is taken as the response action of the wearer of the head mounted device;

[0079] Alternatively, if the relative movement direction is returning to the original position after shaking the head upward and / or returning to the original position after shaking the head downward, a nodding action is used as a response action of the wearer of the head-mounted device.

[0080] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0081] When the wearer's response action is a head shaking action, triggering a rejection processing instruction corresponding to the head shaking action, so as to process the pending incoming call through the rejection processing instruction;

[0082] Alternatively, when the wearer's response action is a head shaking action, a connection processing instruction corresponding to the head shaking action is triggered, so as to process the incoming call to be processed through the connection processing instruction.

[0083] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0084] When the wearer's response action is a nod action, triggering a connection processing instruction corresponding to the nod action, so as to process the incoming call to be processed through the connection processing instruction;

[0085] Alternatively, when the wearer's response action is a nod action, a rejection processing instruction corresponding to the nod action is triggered, so as to process the pending incoming call through the rejection processing instruction.

[0086] In some embodiments, the sensor measurement data includes inertial measurement data, and the inertial measurement data includes three-axis angular velocity data and three-axis acceleration data of the head mounted device. The processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps can be implemented:

[0087] Based on the three-axis angular velocity data and the three-axis acceleration data, obtaining the movement direction angle of the head mounted device at each time point;

[0088] Acquire the time point when the motion direction angle is greater than a preset angle threshold as the turning point time point;

[0089] Determining a reset time point of the head mounted device based on the three-axis angular velocity data and the three-axis acceleration data;

[0090] The relative orientation of the device between the inflection point time point and the reset time point is obtained as the relative movement orientation of the head mounted device.

[0091] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0092] Determine the ground distance change between the inflection point time point and the reset time point according to the distance measurement sensor data;

[0093] The ground distance change is obtained, and the relative orientation of the device between the inflection point time point and the reset time point is determined as the relative movement orientation of the head mounted device.

[0094] In some embodiments, the processor 101 is used to run a computer program stored in the memory 102, and when executing the computer program, the following steps may be implemented:

[0095] When the incoming call to be processed is connected, detecting the number of times the head-mounted device continues to shake its head;

[0096] If the head-mounted device continues shaking its head for more than a preset shaking head threshold, a hang-up processing instruction is executed to hang up the connected incoming call to be processed.

[0097] It should be noted that technical personnel in the relevant field can clearly understand that for the convenience and conciseness of description, the specific working process of the head-mounted device described above can refer to the corresponding process in the following head-mounted device interactive control method embodiment, and will not be repeated here.

[0098] In the following, Figure 1 The head mounted device shown in is taken as an example as the execution subject of the head mounted device interactive control method, and the head mounted device interactive control method provided by the embodiment of the present application is described in detail. For the sake of simplicity and convenience of description, the execution subject will be omitted in the subsequent method embodiments. It should be noted that Figure 1 The scenario is only used to explain the head-mounted device interactive control method provided in the embodiment of the present application, but does not constitute a limitation on the application scenario of the head-mounted device interactive control method provided in the embodiment of the present application.

[0099] See also Figure 2 , Figure 2 201 to 203, wherein:

[0100] 201. When a head mounted device has an incoming call to be processed, obtain sensor measurement data of the head mounted device.

[0101] Among them, the head-mounted device can be smart glasses, smart helmets, etc., the smart glasses can be AR glasses, VR glasses, MR glasses, XR glasses, etc., and the smart helmet can be an AR helmet, VR helmet, MR helmet, XR helmet, etc.

[0102] In some embodiments, the incoming call to be processed may be an incoming call received by call software on the head-mounted device. For example, call software is installed on the head-mounted device, and the call software on the head-mounted device may receive voice or video call messages via the network.

[0103] In other embodiments, the incoming call to be processed may also be an incoming call received by the call software on the terminal device connected to the head-mounted device. For example, taking the head-mounted device as AR glasses and the terminal device as a mobile phone, the AR glasses are connected to the mobile phone via Bluetooth, and the AR glasses can synchronize notification messages on the mobile phone (such as incoming call messages). When the call software of the mobile phone (such as chat software) receives a voice or video call message, it will be synchronized to the AR glasses, and the AR glasses can process the incoming call message synchronized from the mobile phone.

[0104] There are many ways to obtain sensor measurement data in step 201, illustratively including:

[0105] (1) The sensor measurement data includes inertial measurement data. In this case, an inertial measurement unit needs to be set in the head-mounted device to obtain inertial measurement data through the inertial measurement unit when there is a pending call on the head-mounted device. The inertial measurement data may include three-axis acceleration data and three-axis angular velocity data for subsequent analysis of the wearer's response action.

[0106] (2) Sensor measurement data includes inertial measurement data and ranging sensor data. In this case, an inertial measurement unit and a ranging sensor need to be set in the head-mounted device so that when there is a pending incoming call on the head-mounted device, the inertial measurement data is obtained through the inertial measurement unit and the ranging sensor data is obtained through the ranging sensor. The ranging sensor can be an ultrasonic sensor, an infrared sensor, etc.

[0107] 202. Perform identification processing based on the sensor measurement data to obtain the relative movement orientation of the head mounted device.

[0108] Among them, the relative motion orientation is used to indicate the change in the orientation of the head-mounted device relative to the wearer (such as the left, right, top, and bottom of the wearer).

[0109] There are many implementations of step 202, which include, for example:

[0110] (1) In some embodiments, the relative motion orientation is determined based on inertial measurement data. For example, the inertial measurement data includes three-axis angular velocity data and three-axis acceleration data of the head-mounted device. In this case, step 202 may specifically include: inputting the three-axis angular velocity data and three-axis acceleration data of the head-mounted device into a trained orientation recognition model; and performing recognition based on the three-axis angular velocity data and three-axis acceleration data of the head-mounted device using the trained orientation recognition model to obtain the relative motion orientation of the head-mounted device.

[0111] (2) In some embodiments, the relative motion orientation is determined based on inertial measurement data and ranging sensor data. Figure 3 As shown, step 202 may specifically include the following steps 2021 to 2024:

[0112] 2021. Based on the three-axis angular velocity data and the three-axis acceleration data, obtain the movement direction angle of the head-mounted device at each time point.

[0113] The three-axis angular velocity data includes the three-axis angular velocity of the head-mounted device at each time point.

[0114] The three-axis acceleration data includes the three-axis acceleration of the head-mounted device at each time point.

[0115] The motion direction angle at each time point is used to indicate the angle formed between the motion directions of two adjacent time points.

[0116] In some embodiments, the angle of the movement direction at each time point t may be the angle formed between the movement direction at each time point t and the movement direction at the previous time point t-1 of each time point.

[0117] In other embodiments, the angle of the movement direction at each time point t may be the angle formed between the movement direction at each time point t and the movement direction at the next time point t+1 of each time point.

[0118] Exemplarily, in step 2021, first, the three-axis angular velocity at each time point is integrated to calculate the rotation angle of the head-mounted device; then, the angular velocity is converted into a quaternion form to obtain the posture of the head-mounted device at each time point; the three-axis acceleration at each time point is converted to the world coordinate system to obtain the three-axis acceleration at each time point in the world coordinate system; then, the three-axis acceleration at each time point in the world coordinate system and the posture of the head-mounted device at each time point are calculated to obtain the motion direction at each time point. Finally, the motion direction at each time point t and the angle formed between the motion direction at the previous time point t-1 at each time point are calculated as the motion direction angle at each time point t, thereby obtaining the motion direction angle of the head-mounted device at each time point.

[0119] 2022. Obtain a time point at which the motion direction angle is greater than a preset angle threshold as a turning point time point.

[0120] Among them, the specific value of the preset angle threshold can be set according to the actual business scenario requirements, for example, it can be 80°, 90°, etc.

[0121] It is understandable that there may be multiple turning point time points detected in step 2022. For example, when the wearer first turns his head to the left and then returns to the original position, and then turns his head to the right and then returns to the original position, two turning point time points may be detected, namely: time point 1 when the wearer reaches the maximum amplitude position on the left from the original position, and time point 2 when the wearer reaches the maximum amplitude position on the right from the original position.

[0122] 2023. Determine a reset time point of the head mounted device based on the three-axis angular velocity data and the three-axis acceleration data.

[0123] Exemplarily, the time point when the change in the three-axis acceleration is less than a preset first change threshold, and / or the time point when the change in the three-axis angular velocity is less than a preset second change threshold can be used as the reset time point of the head mounted device.

[0124] 2024. Obtain a relative orientation of the device between the inflection point time point and the reset time point as a relative movement orientation of the head mounted device.

[0125] There are many ways to implement step 2024, illustratively including:

[0126] (1) In some embodiments, the method of determining the movement direction of each time point t in step 2021 can be referred to, and the movement direction of the inflection time point and the movement direction of the reset time point can be first determined; then the movement direction of the inflection time point and the movement direction of the reset time point can be used to determine the relative orientation of the device between the inflection time point and the reset time point, so as to serve as the relative movement orientation of the head-mounted device.

[0127] (2) Furthermore, since the inertial sensor data mainly provides acceleration and angular velocity, the acceleration and angular velocity need to be integrated to obtain the position, but the integration process will cause the error to continue to increase. Therefore, the position accuracy derived using the inertial sensor data alone is low. In order to avoid the problem that it is difficult to accurately derive the position using the inertial sensor data alone, and to avoid the problem that it is difficult to accurately estimate the movement direction of the device (such as left, right, up or down) using the inertial sensor data alone, the detection accuracy of the relative movement orientation of the head-mounted device is improved. At this time, the relative orientation of the device can also be determined in combination with the ranging sensor data. The process of detecting the relative orientation of the device in step 2024 can be shown as follows: Steps A1 to A2:

[0128] A1. Determine the ground distance change between the inflection point time point and the reset time point according to the distance measurement sensor data.

[0129] Among them, the ranging sensor data includes the distance between the head-mounted device and the ground at each time point. In this embodiment, the detection distance of the ranging sensor of the head-mounted device at each time point t is used as the distance between the head-mounted device and the ground at each time point t.

[0130] The first ground distance refers to the distance between the head mounted device and the ground at the reset time point, refer to Figure 4 L3 in (a), Figure 4 (a) shows the detection distance L3 of the ranging sensor of the head mounted device at the reset time point.

[0131] The second ground distance refers to the distance between the head mounted device and the ground at the inflection point. Figure 4 L4 in (b), Figure 4 (b) shows the detection distance L4 of the ranging sensor of the head mounted device at the reset time point.

[0132] The ground distance change is a distance change calculated based on the first ground distance and the second ground distance.

[0133] Exemplarily, the head-mounted device may also be provided with a ranging sensor. First, based on the ranging sensor data, a first ground distance between the head-mounted device and the ground at the reset time point is obtained, and a second ground distance between the head-mounted device and the ground at the inflection point time point is obtained; then, the difference between the first ground distance and the second ground distance is calculated as the ground distance change between the inflection point time point and the reset time point.

[0134] A2. Obtain the ground distance change, and determine the relative orientation of the device between the inflection point time point and the reset time point as the relative movement orientation of the head-mounted device.

[0135] For example, if the ground distance change is greater than the first preset distance threshold, the head swinging leftward and / or the head swinging rightward is used as the relative movement orientation of the head mounted device. If the ground distance change is less than the second preset distance threshold, the head swinging upward and / or the head swinging downward is used as the relative movement orientation of the head mounted device.

[0136] Among them, the specific values ​​of the first preset distance threshold and the second preset distance threshold can be set according to the actual business scenario requirements, and the specific values ​​of the first preset distance threshold and the second preset distance threshold are not restricted here.

[0137] Therefore, on the first hand, Figure 4 As shown, when the wearer nods, due to the small change in the device position between the inflection point time point and the reset time point (reference Figure 4 When L1 is mapped to the ground, it will be magnified, which is reflected as the change in ground distance between the turning point time point and the reset time point (reference Figure 4 L2), therefore, on the one hand, even when the amplitude of the wearer's response action is relatively low, it can still be accurately detected, thereby improving the detection accuracy of the wearer's response action;

[0138] Secondly, since the change in ground distance is relatively large when nodding and relatively small (or even no change) when shaking the head, the movement direction of the device (such as left, right, up or down) can be determined by whether the change in ground distance is greater than the first preset distance threshold or less than the second preset threshold. Therefore, the movement direction of the wearer's response action (such as left, right, up or down) can be accurately detected, and there is no need to perform error correction on the inertial sensor data to analyze the movement direction, thereby avoiding the problem of low position accuracy derived from inertial sensor data alone due to the fact that the inertial sensor data mainly provides acceleration and angular velocity, and reducing the difficulty of inertial sensor data analysis.

[0139] Thirdly, since the "movement direction from the original position to the maximum amplitude position" and the "movement direction from the maximum amplitude position to the original position at the moment" of the wearer nodding or shaking his head are basically in opposite directions, it can be seen that the direction change of "the movement direction from the original position to the maximum amplitude position" and "the movement direction from the maximum amplitude position to the original position at the moment" is relatively obvious. Therefore, no precise direction information is required when determining the turning point time point. The position information calculated using acceleration and angular velocity is sufficient to reflect the direction change at the turning point time point, and error correction is not required, thereby reducing the difficulty of detecting the relative motion orientation of the head-mounted device.

[0140] 203. Determine a response action of a wearer of the head mounted device based on the relative motion orientation.

[0141] The wearer response action is a response action made by the wearer of the head mounted device to the incoming call to be processed, which is determined based on the sensor measurement data. The wearer response action can control the triggering of the corresponding processing instruction. For example, when the wearer response action is shaking the head, the rejection processing instruction is triggered, and when the wearer response action is nodding, the connection processing instruction is triggered.

[0142] Exemplarily, at this time, step 203 may specifically include the following steps 2031A to 2032A:

[0143] 2031A. If the relative movement direction is returning to the original position after shaking the head to the left and / or returning to the original position after shaking the head to the right, the shaking head action is used as the response action of the wearer of the head-mounted device.

[0144] For example, in some embodiments, if the relative movement orientation is shaking the head to the left and then returning to the original position, the shaking head motion is taken as a response motion of the wearer of the head mounted device.

[0145] For another example, in some embodiments, if the relative movement direction is shaking the head to the right and then returning to the original position, the shaking head action is regarded as a response action of the wearer of the head mounted device.

[0146] For another example, in some embodiments, if the relative movement orientation is first shaking the head to the right and then returning to the original position, and then shaking the head to the left and then returning to the original position, then the shaking head motion is used as a response motion of the wearer of the head mounted device.

[0147] For another example, in some embodiments, if the relative movement orientation is first shaking the head to the left and then returning to the original position, and then shaking the head to the right and then returning to the original position, then the shaking head motion is used as a response motion of the wearer of the head mounted device.

[0148] 2032A. If the relative movement direction is returning to the original position after shaking the head upward and / or returning to the original position after shaking the head downward, a nodding action is used as a response action of the wearer of the head-mounted device.

[0149] For example, in some embodiments, if the relative movement orientation is to swing the head upward and then return to the original position, a nodding action is taken as a response action of the wearer of the head mounted device.

[0150] For another example, in some embodiments, if the relative movement orientation is to move the head downward and then return to the original position, a nodding action is taken as a response action of the wearer of the head mounted device.

[0151] For another example, in some embodiments, if the relative movement orientation is first shaking the head upward and then returning to the original position, and then shaking the head downward and then returning to the original position, then a nodding action is used as a response action of the wearer of the head mounted device.

[0152] For another example, in some embodiments, if the relative movement orientation is first shaking the head downward and then returning to the original position, and then shaking the head upward and then returning to the original position, then the nodding action is used as the response action of the wearer of the head mounted device.

[0153] 204. Trigger a target processing instruction corresponding to the wearer's response action, so as to process the incoming call to be processed through the target processing instruction.

[0154] (1) For example, the processing instruction corresponding to the shaking head action is a rejection processing instruction, and the processing instruction corresponding to the nodding action is a connection processing instruction. In this case, in step 203, when the wearer's response action is a shaking head action, the rejection processing instruction corresponding to the shaking head action is triggered, so that the pending incoming call is rejected through the rejection processing instruction. When the wearer's response action is a nodding action, the connection processing instruction corresponding to the nodding action is triggered, so that the pending incoming call is connected through the connection processing instruction.

[0155] For example, taking the head-mounted device as AR glasses, when there is an incoming call on the AR glasses, the wearer first shakes his head to the left and then returns to the original position, and then shakes his head to the right and then returns to the original position. At this time, step 201 analyzes the sensor measurement data collected by the head-mounted device to determine that the wearer's response action is a head shake action. At this time, a rejection processing instruction corresponding to the head shake action will be triggered to reject the pending incoming call of the AR glasses through the rejection processing instruction.

[0156] For another example, taking the head-mounted device as AR glasses, when there is an incoming call on the AR glasses, the wearer first shakes his head upward and then returns to the original position, and then shakes his head downward and then returns to the original position. At this time, step 201 analyzes the sensor measurement data collected by the head-mounted device to determine that the wearer's response action is a nod action. At this time, the connection processing instruction corresponding to the nod action will be triggered, so as to connect the pending incoming call of the AR glasses through the connection processing instruction.

[0157] Therefore, by triggering a rejection processing instruction corresponding to a shaking head action, or triggering a connection processing instruction corresponding to a nodding action, the incoming call reminder of the head-mounted device can be processed when the user's hands are unable to be freed.

[0158] (2) For example, the processing instruction corresponding to the shaking head action is a connection processing instruction, and the processing instruction corresponding to the nodding action is a rejection processing instruction. In this case, in step 203, when the wearer's response action is a shaking head action, the connection processing instruction corresponding to the shaking head action is triggered, so that the pending incoming call is connected through the connection processing instruction. When the wearer's response action is a nodding action, the rejection processing instruction corresponding to the nodding action is triggered, so that the pending incoming call is rejected through the rejection processing instruction.

[0159] For example, taking the head-mounted device as AR glasses, when there is an incoming call on the AR glasses, when the wearer shakes his head to the left and returns to the original position (or when the wearer shakes his head to the right and returns to the original position), step 201 analyzes the sensor measurement data collected by the head-mounted device to determine that the wearer's response action is a head shake action. At this time, the connection processing instruction corresponding to the head shake action will be triggered to connect the pending incoming call of the AR glasses through the connection processing instruction.

[0160] For another example, taking the head-mounted device as AR glasses, when there is an incoming call on the AR glasses, the wearer first shakes his head upward and then returns to the original position (or shakes his head downward and then returns to the original position), at this time, step 201 analyzes the sensor measurement data collected by the head-mounted device to determine that the wearer's response action is a nod action. At this time, the rejection processing instruction corresponding to the nod action will be triggered, so as to reject the pending incoming call of the AR glasses through the rejection processing instruction.

[0161] Therefore, by triggering a rejection processing instruction corresponding to a nodding action, or triggering a connection processing instruction corresponding to a shaking head action, the incoming call reminder of the head-mounted device can be processed when the user's hands are unable to be freed.

[0162] From the above content, it can be seen that when there is a pending incoming call on the head-mounted device, the inertial measurement data of the head-mounted device is obtained for identification and processing, so as to obtain the relative movement orientation of the head-mounted device, which is used to determine the response action of the wearer of the head-mounted device, trigger the target processing instruction corresponding to the wearer's response action, and process the pending incoming call through the target processing instruction; thereby, the user can process the incoming call reminder of the head-mounted device through head movement, and realize the processing of the incoming call reminder when the user's hands cannot be freed, thereby improving the convenience of processing the incoming call reminder of the head-mounted device, and thus improving the user experience.

[0163] Furthermore, when the pending call is connected, the number of times the head-mounted device shakes its head is continuously moved is detected; if the number of times the head-mounted device shakes its head is continuously moved is greater than a preset shaking number threshold, a hang-up processing instruction is executed to hang up the pending call that has been connected. In this way, when the call is connected and both hands cannot be freed, the connected call can be hung up, thereby improving the call processing efficiency and the convenience of operation.

[0164] In addition, in order to better implement the head-mounted device interactive control method in the embodiment of the present application, based on the head-mounted device interactive control method, the embodiment of the present application also provides a head-mounted device interactive control device, such as Figure 5 As shown, a schematic structural diagram of an embodiment of a head mounted device interactive control device provided in an embodiment of the present application is provided, and an identification unit 502 is provided. The head mounted device interactive control device 500 includes:

[0165] An acquisition unit 501 is used to acquire sensor measurement data of the head mounted device when there is a pending incoming call on the head mounted device;

[0166] An identification unit 502 is used to perform identification processing based on the sensor measurement data to obtain the relative motion orientation of the head mounted device;

[0167] A determination unit 503, configured to determine a response action of a wearer of the head mounted device based on the relative motion orientation;

[0168] The processing unit 504 is used to trigger a target processing instruction corresponding to the wearer's response action, so as to process the incoming call to be processed through the target processing instruction.

[0169] In some embodiments, the determining unit 503 is specifically configured to:

[0170] If the relative movement direction is returning to the original position after shaking the head to the left and / or returning to the original position after shaking the head to the right, the shaking head action is taken as the response action of the wearer of the head mounted device;

[0171] Alternatively, if the relative movement direction is returning to the original position after shaking the head upward and / or returning to the original position after shaking the head downward, a nodding action is used as a response action of the wearer of the head-mounted device.

[0172] In some embodiments, the processing unit 504 is specifically configured to:

[0173] When the wearer's response action is a head shaking action, triggering a rejection processing instruction corresponding to the head shaking action, so as to process the pending incoming call through the rejection processing instruction;

[0174] Alternatively, when the wearer's response action is a head shaking action, a connection processing instruction corresponding to the head shaking action is triggered, so as to process the incoming call to be processed through the connection processing instruction.

[0175] In some embodiments, the processing unit 504 is specifically configured to:

[0176] When the wearer's response action is a nod action, triggering a connection processing instruction corresponding to the nod action, so as to process the incoming call to be processed through the connection processing instruction;

[0177] Alternatively, when the wearer's response action is a nod action, a rejection processing instruction corresponding to the nod action is triggered, so as to process the pending incoming call through the rejection processing instruction.

[0178] In some embodiments, the inertial measurement data includes three-axis angular velocity data and three-axis acceleration data of the head mounted device, and the identification unit 502 is specifically used for:

[0179] Based on the three-axis angular velocity data and the three-axis acceleration data, obtaining the movement direction angle of the head mounted device at each time point;

[0180] Acquire the time point when the motion direction angle is greater than a preset angle threshold as the turning point time point;

[0181] Determining a reset time point of the head mounted device based on the three-axis angular velocity data and the three-axis acceleration data;

[0182] The relative orientation of the device between the inflection point time point and the reset time point is obtained as the relative movement orientation of the head mounted device.

[0183] In some embodiments, the sensor measurement data further includes ranging sensor data, and the identification unit 502 is specifically configured to:

[0184] Determine the ground distance change between the inflection point time point and the reset time point according to the distance measurement sensor data;

[0185] The ground distance change is obtained, and the relative orientation of the device between the inflection point time point and the reset time point is determined as the relative movement orientation of the head mounted device.

[0186] In some embodiments, the processing unit 504 is specifically configured to:

[0187] When the incoming call to be processed is connected, detecting the number of times the head-mounted device continues to shake its head;

[0188] If the head-mounted device continues shaking its head for more than a preset shaking head threshold, a hang-up processing instruction is executed to hang up the connected incoming call to be processed.

[0189] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can refer to the previous embodiment of the head-mounted device interactive control method, which will not be repeated here.

[0190] A person of ordinary skill in the art will appreciate that all or part of the steps in the above-mentioned head-mounted device interactive control method may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0191] To this end, an embodiment of the present application provides a computer-readable storage medium, in which multiple computer programs are stored. The computer programs can be loaded by a processor to execute any head-mounted device interactive control method provided in the embodiment of the present application.

[0192] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0193] In the above-mentioned embodiments of the head-mounted device interactive control device, computer-readable storage medium, and head-mounted device, the description of each embodiment has its own emphasis. For the parts not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process and beneficial effects of the head-mounted device interactive control device, computer-readable storage medium, head-mounted device and its corresponding units described above can refer to the description of the head-mounted device interactive control method in the above embodiment, and will not be repeated here.

[0194] The above is a detailed introduction to a head-mounted device interactive control method, device, head-mounted device and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scopes. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A head mounted device interactive control method, characterized in that: The method comprises: When the head mounted device has an incoming call to be processed, obtaining sensor measurement data of the head mounted device; Performing identification processing based on the sensor measurement data to obtain the relative motion orientation of the head mounted device; Determining a response action of a wearer of the head mounted device based on the relative motion orientation; A target processing instruction corresponding to the wearer's response action is triggered to process the incoming call to be processed through the target processing instruction.

2. The head mounted device interactive control method according to claim 1, characterized in that: The step of determining a response action of a wearer of the head mounted device based on the relative motion orientation includes: If the relative movement direction is returning to the original position after shaking the head to the left and / or returning to the original position after shaking the head to the right, the shaking head action is taken as the response action of the wearer of the head mounted device; Alternatively, if the relative movement direction is returning to the original position after shaking the head upward and / or returning to the original position after shaking the head downward, a nodding action is used as a response action of the wearer of the head-mounted device.

3. The head mounted device interactive control method according to claim 2, characterized in that: The triggering of the target processing instruction corresponding to the wearer's response action, so as to process the incoming call to be processed by the target processing instruction, includes: When the wearer's response action is a head shaking action, triggering a rejection processing instruction corresponding to the head shaking action, so as to process the pending incoming call through the rejection processing instruction; Alternatively, when the wearer's response action is a head shaking action, a connection processing instruction corresponding to the head shaking action is triggered, so as to process the incoming call to be processed through the connection processing instruction.

4. The head mounted device interactive control method according to claim 2, characterized in that: The triggering of the target processing instruction corresponding to the wearer's response action, so as to process the incoming call to be processed by the target processing instruction, includes: When the wearer's response action is a nod action, triggering a connection processing instruction corresponding to the nod action, so as to process the incoming call to be processed through the connection processing instruction; Alternatively, when the wearer's response action is a nod action, a rejection processing instruction corresponding to the nod action is triggered, so as to process the pending incoming call through the rejection processing instruction.

5. The head mounted device interactive control method according to claim 1, characterized in that: The sensor measurement data includes inertial measurement data, and the inertial measurement data includes three-axis angular velocity data and three-axis acceleration data of the head-mounted device. The identification processing based on the sensor measurement data to obtain the relative motion orientation of the head-mounted device includes: Based on the three-axis angular velocity data and the three-axis acceleration data, obtaining the movement direction angle of the head mounted device at each time point; Acquire the time point when the motion direction angle is greater than a preset angle threshold as the turning point time point; Determining a reset time point of the head mounted device based on the three-axis angular velocity data and the three-axis acceleration data; The relative orientation of the device between the inflection point time point and the reset time point is obtained as the relative movement orientation of the head mounted device.

6. The head mounted device interactive control method according to claim 5, characterized in that: The sensor measurement data also includes distance measurement sensor data, and obtaining the relative orientation of the device between the inflection point time point and the reset time point as the relative movement orientation of the head mounted device includes: Determine the ground distance change between the inflection point time point and the reset time point according to the distance measurement sensor data; The ground distance change is obtained, and the relative orientation of the device between the inflection point time point and the reset time point is determined as the relative movement orientation of the head mounted device.

7. The head mounted device interactive control method according to claim 1, characterized in that: The method further comprises: When the incoming call to be processed is connected, detecting the number of times the head-mounted device continues to shake its head; If the head-mounted device continues shaking its head for more than a preset shaking head threshold, a hang-up processing instruction is executed to hang up the connected incoming call to be processed.

8. A head mounted device interactive control device, characterized in that: The head mounted device interactive control device comprises: An acquisition unit, configured to acquire sensor measurement data of the head mounted device when there is an incoming call to be processed on the head mounted device; An identification unit, configured to perform identification processing based on the sensor measurement data to obtain a relative motion orientation of the head mounted device; a determining unit, configured to determine a response action of a wearer of the head mounted device based on the relative motion orientation; A processing unit is used to trigger a target processing instruction corresponding to the wearer's response action, so as to process the incoming call to be processed through the target processing instruction.

9. A head mounted device, characterized in that: It comprises a processor and a memory, wherein the memory stores a computer program, and when the processor calls the computer program in the memory, the head mounted device interactive control method as claimed in any one of claims 1 to 7 is executed.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the head mounted device interactive control method according to any one of claims 1 to 7.

Images