Method and device for determining holding mode, terminal equipment and storage medium
By obtaining inertial sensor data and feature data, and using neural network algorithms to determine the grip mode of folding screen terminal equipment, the problem that the user's grip mode affects the performance of the device is solved, and accurate grip mode judgment and equipment performance optimization are achieved, with low Power consumption and wide applicability.
Patent Information
- Application Number
- CN202310736003.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-20
- Publication Date
- 2024-12-20
AI Technical Summary
When using folding screen terminal devices, the user's grip method has an impact on the performance of the equipment, and it is difficult for the prior art to accurately judge and adapt to different grip methods, resulting in poor signal.
By acquiring inertial sensor data and characteristic data, a neural network algorithm is used to determine the grip of the terminal device, including left-hand, right-hand, and two-hand grip, and the final grip is determined based on the probability value and threshold value to optimize the equipment performance.
It realizes accurate grasping of the terminal device, optimizes anti-detect and signal enhancement functions, and only relies on inertial sensors, with low power consumption and wide applicability.
Smart Images
Figure CN119172469A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of smart terminals, and in particular to a method, device, terminal equipment and storage medium for determining a holding mode. Background Art
[0002] When a user uses a foldable screen terminal device, the user can use it as a normal screen terminal device when the foldable screen is folded, or use it as a large screen terminal device when the foldable screen is unfolded. When a user uses a foldable screen terminal device in the unfolded state, holding the terminal device with different hands will have different effects on the terminal device. For example, when holding the terminal device with the left hand and the right hand, different anti-mistouch strategies need to be used, and the use strategies of the signal transmitting antenna and the signal receiving antenna in the terminal device need to be adjusted to avoid poor signals caused by hand occlusion. Therefore, when a user uses a foldable screen terminal device, it is crucial to judge how the user holds the terminal device. Summary of the invention
[0003] In order to overcome the problems existing in the related art, the present disclosure provides a method, apparatus, terminal device and storage medium for determining a holding manner.
[0004] According to a first aspect of an embodiment of the present disclosure, a method for determining a holding manner is provided, the method comprising:
[0005] Acquire inertial sensor data and / or feature data within a preset time period, wherein the inertial sensor data includes at least one of an inertial sensor reading and a statistical value of the inertial sensor reading, and the feature data includes at least one of time domain feature data, frequency domain feature data, and spectrum feature data of the inertial sensor data;
[0006] Obtaining, according to the inertial sensor data and / or the characteristic data, a first holding mode of the terminal and a probability value of the first holding mode, wherein the first holding mode includes at least one of the following: left-hand holding, right-hand holding, and both-hand holding;
[0007] A second holding way is determined according to the first holding way and a probability value of the first holding way, and the second holding way is determined as the holding way of the terminal.
[0008] In an exemplary embodiment, obtaining the first holding manner of the terminal and the probability value of the first holding manner according to the inertial sensor data and / or the feature data includes:
[0009] determining a neural network algorithm based on at least one of a memory size of the terminal, a placement of the terminal, and a position of the inertial sensor in the terminal;
[0010] The inertial sensor data and / or the feature data are input into the neural network algorithm to obtain the first holding manner and the probability value of the first holding manner.
[0011] In an exemplary embodiment, determining the second holding way according to the first holding way and the probability value of the first holding way includes:
[0012] If the probability value of the first holding way is greater than or equal to a first threshold, determining that the first holding way is the second holding way;
[0013] If the probability value of the first holding method is less than the first threshold, and the number of acquisitions of the same type of the first holding method is greater than or equal to a second threshold, the first holding method is determined to be the second holding method.
[0014] In an exemplary embodiment, determining the second holding way according to the first holding way and the probability value of the first holding way includes:
[0015] If the first holding manner is the same as the previous second holding manner, determining the previous second holding manner to be the second holding manner;
[0016] If the first holding manner is different from the previous second holding manner, the second holding manner is determined according to the first holding manner obtained within a first preset time period.
[0017] In an exemplary embodiment, determining the second holding manner according to the first holding manner obtained within a first preset time period includes:
[0018] In the first preset time period, if the first holding manner is the same as the previous second holding manner, determining the previous second holding manner to be the second holding manner;
[0019] If the first holding way is not the same as the previous second holding way, it is determined that the first holding way is the second holding way.
[0020] In an exemplary embodiment, determining the second holding way according to the first holding way and the probability value of the first holding way includes:
[0021] At least two probability values of the first holding manner and the first holding manner are obtained within a second preset time period, and the first holding manner corresponding to the maximum value of the probability values is determined as the second holding manner.
[0022] In an exemplary embodiment, the method further comprises:
[0023] The updating frequency of the holding way of the terminal is determined according to the application scenario of the holding way of the terminal.
[0024] In an exemplary embodiment, the inertial sensor includes an acceleration sensor and an angular velocity sensor.
[0025] According to a second aspect of an embodiment of the present disclosure, a device for determining a holding mode is provided, the device comprising:
[0026] an acquisition module, configured to acquire inertial sensor data and / or feature data within a preset time period, wherein the inertial sensor data includes at least one of an inertial sensor reading and a statistical value of the inertial sensor reading, and the feature data includes at least one of time domain feature data, frequency domain feature data, and spectrum feature data of the inertial sensor data;
[0027] a calculation module, configured to obtain a first holding mode of the terminal and a probability value of the first holding mode according to the inertial sensor data and / or the feature data, wherein the first holding mode includes at least one of the following: left-hand holding, right-hand holding, and both-hand holding;
[0028] The determination module is configured to determine a second holding method according to the first holding method and a probability value of the first holding method, and determine the second holding method as the holding method of the terminal.
[0029] According to a third aspect of an embodiment of the present disclosure, a terminal device is provided, including:
[0030] processor;
[0031] a memory for storing processor-executable instructions;
[0032] The processor is configured to execute the method as described in the first aspect of the embodiment of the present disclosure.
[0033] According to a fourth aspect of an embodiment of the present disclosure, a non-temporary computer-readable storage medium is provided. When instructions in the storage medium are executed by a processor of a terminal device, the terminal device is enabled to execute the method described in the first aspect of the embodiment of the present disclosure.
[0034] The above method of the present invention has the following beneficial effects: the method of the present invention can accurately obtain the holding method of the terminal, and obtain relevant data under the terminal usage status only through the inertial sensor, without relying on other hardware structures, and has the characteristics of low power consumption and wide applicability.
[0035] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.
[0037] Figure 1 is a flow chart of a method for determining a holding mode according to an exemplary embodiment;
[0038] Figure 2A is a schematic diagram of a space coordinate system of a terminal device according to an exemplary embodiment;
[0039] Figure 2B is a schematic diagram showing the position of an inertial sensor in a terminal device according to an exemplary embodiment;
[0040] Figure 3A is a schematic diagram showing a terminal device held in the left hand according to an exemplary embodiment;
[0041] Figure 3B is a schematic diagram showing a right hand holding a terminal device according to an exemplary embodiment;
[0042] Figure 3C is a schematic diagram showing a terminal device held with two hands according to an exemplary embodiment;
[0043] Figure 4 is a schematic diagram of a neural network algorithm structure according to an exemplary embodiment;
[0044] Figure 5 is a block diagram of a device for determining a holding mode according to an exemplary embodiment;
[0045] Figure 6 It is a block diagram of a terminal device for executing a method for determining a holding manner according to an exemplary embodiment. DETAILED DESCRIPTION
[0046] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.
[0047] In an exemplary embodiment of the present disclosure, a method for determining a holding method is provided, which is applied to a terminal, and inertial sensor data and / or feature data within a preset time length are obtained. According to the inertial sensor data and / or feature data, a first holding method of the terminal and a probability value of the first holding method are obtained, wherein the first holding method includes at least one of the following: left-hand holding, right-hand holding, and both-hand holding. According to the first holding method and the probability value of the first holding method, a second holding method is determined, and the second holding method is determined as the holding method of the terminal. The method in the present disclosure can accurately obtain the holding method of the terminal to optimize the extended functions related to the holding method in the terminal, and only obtains relevant data in the terminal use state through the inertial sensor, without relying on other hardware structures, and has the characteristics of low power consumption and wide applicability.
[0048] The method for determining the holding mode provided by the present disclosure can be applied to terminal devices with any display screen size, including mobile phones with ordinary display screen size, large-screen mobile phones, unfolded folding screen mobile phones, folded folding screen mobile phones, tablets with any display screen size, and other electronic devices. In the exemplary embodiment of the present disclosure, the terminal device is described by taking an unfolded folding screen mobile phone as an example.
[0049] In an exemplary embodiment of the present disclosure, a method for determining a holding manner is provided, which is applied to a terminal. Figure 1 is a flow chart of a method for determining a holding mode according to an exemplary embodiment. Figure 1 As shown, the following steps are included:
[0050] Step S101, acquiring inertial sensor data and feature data within a preset time period.
[0051] The preset time for obtaining inertial sensor data and feature data is an empirical value, which is determined according to actual needs. For example, the preset time is 3 seconds or 0.8 seconds, or the inertial sensor data and feature data are obtained within 0.8 seconds and 3 seconds respectively. When the terminal device is in a stationary state, obtaining data within a shorter preset time can increase the speed of determining the holding method. However, when the terminal device is in motion, the data obtained in a shorter preset time may be inaccurate. At this time, the data obtained in a longer preset time can be used to correct the impact of the motion state, which can improve the accuracy of the results.
[0052] The frequency of acquiring data within the preset time is an empirical value and is determined according to actual needs. For example, when acquiring data within 0.8 seconds, a frequency of 50 Hz is used, and a total of 0.8×50=40 data are acquired; when acquiring data within 3 seconds, a frequency of 25 Hz is used, and a total of 3×25=75 data are acquired.
[0053] The inertial sensor data includes at least one of an inertial sensor reading and a statistical value of the inertial sensor reading, and the feature data includes at least one of time domain feature data, frequency domain feature data, and spectrum feature data of the inertial sensor data.
[0054] Inertial sensors (IMU, Inertial measurement unit) are used to measure the angular velocity and acceleration of terminal devices in the x-axis, y-axis, and z-axis of the spatial coordinate system. Inertial sensors are mainly composed of acceleration sensors and angular velocity sensors, such as accelerometers and gyroscopes. Figure 2A is a schematic diagram of a space coordinate system of a terminal device according to an exemplary embodiment. Figure 2A As shown, a spatial coordinate system is established with the center point of the terminal device as the origin. Figure 2B is a schematic diagram showing the position of an inertial sensor in a terminal device according to an exemplary embodiment, such as Figure 2B As shown, inertial sensors 201, 202 and 203 are used to obtain the acceleration and angular velocity of the terminal device on the x-axis, y-axis and z-axis.
[0055] The inertial sensor readings are the acceleration values of the three axes and the angular velocity values of the three axes. The statistical values of the inertial sensor readings include the mean of the three-axis acceleration values, the mean of the three-axis angular velocity values, the maximum of the three-axis acceleration values, the maximum of the three-axis angular velocity values, the difference of the three-axis acceleration values, and the difference of the three-axis angular velocity values. The difference of the acceleration values and the difference of the angular velocity values represent the differences between the acceleration value and the angular velocity value obtained at the current moment and the acceleration value and the angular velocity value obtained at the previous moment, respectively. In addition, the three-axis acceleration values can be filtered through an infinite impulse response (IIR) filter to obtain the three-axis gravity values, and the three-axis acceleration values are respectively subtracted from the three-axis gravity values to obtain the three-axis linear acceleration values. Therefore, the inertial sensor readings can also include the three-axis gravity values and the three-axis linear acceleration values. The statistical values of the inertial sensor readings can also include the mean of the three-axis gravity values, the mean of the three-axis linear acceleration values, the maximum value of the three-axis gravity values, the maximum value of the three-axis linear acceleration values, the difference of the three-axis gravity values, and the difference of the three-axis linear acceleration values. The inertial sensor data includes at least one of the above-mentioned inertial sensor readings and the statistical values of the above-mentioned inertial sensor readings. It can be understood that the more types of data included, the more accurate the determined holding method is.
[0056] Each time the inertial sensor data is acquired within a preset time length, the characteristic data of the inertial sensor data is calculated. The time domain characteristic data of the inertial sensor data includes the mean, standard deviation, maximum value, minimum value, mode, value range, number of zero crossings, and sum of all the inertial sensor data acquired within the preset time length at the current moment. The time domain data of all the inertial sensor data acquired within the preset time length at the current moment is converted into frequency domain data. The frequency domain characteristic data of the inertial sensor data includes the DC component in the frequency domain data, the maximum value in the frequency domain, the position of the maximum value, the size of the first 5 peak values, the position of the first 5 peak values, the frequency domain energy, the proportion of spectrum energy from 0 to 3.75 Hz, the proportion of spectrum energy from 3.75 to 8.75 Hz, the proportion of spectrum energy from 8.75 to 12.5 Hz, and the ratio of the maximum value of the spectrum to the DC component of the spectrum. The frequency spectrum feature data of the inertial sensor data includes the frequency spectrum shape data of the frequency domain data and the waveform data of the time domain data. The frequency spectrum shape data of the frequency domain data includes the frequency spectrum shape mean, the frequency spectrum shape standard deviation, the frequency spectrum shape slope, and the frequency spectrum shape kurtosis. The waveform data of the time domain data includes the waveform mean, the waveform standard deviation, the waveform slope, and the waveform kurtosis. The feature data includes at least one of the time domain feature data, the frequency domain feature data, and the frequency spectrum feature data of the inertial sensor data. It can be understood that the more types of data included, the more accurate the determined holding method is.
[0057] Step S102: Obtain a first holding mode of the terminal and a probability value of the first holding mode according to the inertial sensor data and the feature data.
[0058] The first holding method includes at least one of the following: left-hand holding, right-hand holding, and both-hand holding. Figure 3A to Figure 3C Schematic diagrams of holding the terminal device in the left hand, holding the terminal device in the right hand, and holding the terminal device in both hands are respectively shown.
[0059] According to the inertial sensor data and the feature data, the first holding mode of the terminal and the probability value of the first holding mode are predicted, and the probability value indicates the possibility of the first holding mode occurring. For example, if the result obtained is 80% for left-hand holding, the first holding mode is left-hand holding, and the probability value is 80%. The first holding mode and its probability value are obtained by a prediction algorithm, and the prediction algorithm can be a traditional machine learning algorithm, such as a logistic regression algorithm, a support vector machine algorithm, etc., or a deep learning neural network algorithm.
[0060] Inertial sensor data and feature data when holding the terminal device with the left hand, the right hand, and both hands are collected as training data sets to train the preset algorithm. In order to ensure the accuracy of the prediction of the preset algorithm in different states, when collecting the training data sets, the data of holding the terminal device in a static state and the data of holding the terminal device in a moving state are collected respectively. The data in the static state includes the data when the holding hand is in the postures of clicking, sliding, turning the wrist, and small vibration, as well as the data when the terminal device is at various angles facing up and facing up, and the data in the moving state includes the data when walking, running, taking the subway, bus, high-speed rail, airplane, and other means of transportation. After obtaining the training data set, the inertial sensor data and all the feature data can be used as the input of the prediction algorithm to train the preset algorithm, or the inertial sensor data and some feature data selected from all the feature data can be used as the input of the prediction algorithm to train the prediction algorithm. When selecting part of the feature data from all the feature data, for example, calculating the information gain of all the feature data, select part of the feature data with larger information gain values to train the prediction algorithm, for example, select 200 feature data with larger information gain values; or separate a small part of the data from the training data set as a test data set, randomly select multiple groups of feature data in the training data set as inputs of the prediction algorithm to train the prediction algorithm, and then use the test data set to test the prediction algorithms trained by each group of feature data, obtain the accuracy of the prediction algorithms trained by each group of feature data, and use the prediction algorithm trained by a group of feature data with higher accuracy as the prediction algorithm to be used in the end.
[0061] The trained prediction algorithm is pre-stored in the memory of the terminal device, and according to the input of the trained prediction algorithm, the required inertial sensor data and feature data are obtained when the user uses the terminal device, and the first holding method and the probability value of the first holding method are obtained based on the trained prediction algorithm.
[0062] In some possible implementations, when the prediction algorithm is a deep learning neural network algorithm, the neural network algorithm is determined based on at least one of the memory size of the terminal, the placement of the terminal, and the position of the inertial sensor in the terminal, and the inertial sensor data and feature data are input into the neural network algorithm to obtain the first holding method and the probability value of the first holding method.
[0063] Since running a more complex neural network algorithm requires a larger memory space, while running a simpler neural network algorithm requires a smaller memory space, determining the neural network algorithm according to the terminal memory size can make the holding method determination method in the present disclosure applicable to terminal devices with different memory sizes. When the terminal memory is small, a simpler neural network algorithm is selected to obtain the first holding method and its probability value, and at the same time, the feature data used for the neural network algorithm input can be reduced. Figure 4 The schematic diagram of the neural network algorithm structure is shown in Figure 4 As shown in FIG. 1 , the inertial sensor data and some feature data are used as inputs of the neural network algorithm. The inertial sensor data passes through one convolution layer (conv) with a convolution kernel of 1×1 and three convolution layers (conv) with a convolution kernel of 3×3. Some feature data passes through three convolution layers (conv) with a convolution kernel of 3×3. The intermediate feature values (n1 and n2) obtained from the two inputs are fused (concat). The fused intermediate feature values pass through three fully connected layers (FC) for classification, and the first holding method and its probability value are output. When the terminal memory is large, a more complex neural network algorithm is selected to obtain the first holding method and its probability value to improve the accuracy of the result. For example, Figure 4 On the basis of the neural network algorithm structure, attention modules, RepVGG modules, etc. are added to enhance the learning ability of the neural network algorithm. At the same time, more feature data can be used as the input of the neural network algorithm.
[0064] The placement of the terminal indicates the direction in which the content is displayed in the terminal. The four sides of the terminal are marked as 1, 2, 3, and 4. Figure 3A to Figure 3C The placement of the terminal shown is based on the side numbered 1 as the bottom side. Since the inertial sensor data will be different when the terminal is placed in different ways, the placement of the terminal is also a factor that affects the output result of the neural network algorithm. When determining the neural network algorithm, the placement of the terminal is first obtained, and the corresponding neural network algorithm is selected according to the placement, and the inertial sensor data and feature data are input into the neural network algorithm to obtain the first holding method and its probability value.
[0065] Since the position of the inertial sensor in the terminal is different in different models, resulting in different inertial sensor data, in order to improve the universality of the method for determining the holding method in the present disclosure, it is applicable to different models, and the neural network algorithm is determined according to the position of the inertial sensor in the terminal, and the inertial sensor data and / or feature data are input into the neural network algorithm to obtain the first holding method and its probability value. For example, by analyzing the influence of the inertial sensor data on the output result when the position of the inertial sensor in the terminal is different, it is determined that the inertial sensor data that has a greater influence on the output result is the acceleration of the y-axis and the angular velocity of the z-axis. Therefore, in order to improve universality, an attention module can be added to the result of the neural network algorithm to reduce the weight of the acceleration of the y-axis and the angular velocity of the z-axis, thereby reducing the influence of the position of the inertial sensor in the terminal on the output result.
[0066] It is understandable that in order to make the output result of the neural network algorithm more adaptable to the current state of the terminal, the neural network algorithm can be determined based on the memory size of the terminal, the placement of the terminal and the position of the inertial sensor in the terminal.
[0067] In some possible implementations, in the above steps S101 and S102, only inertial sensor data may be obtained, and based on the inertial sensor data, the first holding method of the terminal and the probability value of the first holding method may be obtained; or only feature data may be obtained, and based on the feature data, the first holding method of the terminal and the probability value of the first holding method may be obtained.
[0068] It can be understood that when obtaining the first holding mode and the probability value of the first holding mode of the terminal according to the inertial sensor data, the prediction algorithm is trained by collecting the inertial sensor data as the training set of the prediction algorithm, and the inertial sensor data is obtained as the input of the trained prediction algorithm in actual use, and the first holding mode and the probability value of the first holding mode of the terminal are obtained by output; when obtaining the first holding mode and the probability value of the first holding mode of the terminal according to the feature data, the prediction algorithm is trained by collecting the feature data as the training set of the prediction algorithm, and the feature data is obtained as the input of the trained prediction algorithm in actual use, and the first holding mode and the probability value of the first holding mode of the terminal are obtained by output. For specific implementation methods, see the above steps S101 and S102.
[0069] Step S103: determining a second holding way according to the first holding way and the probability value of the first holding way, and determining the second holding way as the holding way of the terminal.
[0070] The first holding mode is an instantaneous state, and the second holding mode is a continuous state. Since the instantaneous state is unstable, a large error may occur when the first holding mode obtained once is used as the holding mode of the terminal, while the continuous state is stable. Therefore, the second holding mode can be determined by the probability value of the first holding mode and the number of times the first holding mode appears or the change between the first holding mode and the holding mode determined at the previous moment, and the second holding mode is determined as the holding mode of the terminal to improve the accuracy of the result.
[0071] In some possible implementations, the method for determining the second holding manner includes at least one of the following three methods:
[0072] The first one is that if the probability value of the first holding method is greater than or equal to the first threshold, the first holding method is determined to be the second holding method; if the probability value of the first holding method is less than the first threshold, and the number of times the same type of first holding method is obtained is greater than or equal to the second threshold, the first holding method is determined to be the second holding method.
[0073] The first threshold is an empirical value and can be determined according to actual conditions, for example, the first threshold is 90% or 95%. When the probability value of the first holding method is greater than or equal to the first threshold, it means that the obtained first holding method has a high confidence level, and the obtained first holding method is considered to be a continuous state and stable, so the first holding method is determined as the second holding method. When the probability value of the first holding method is less than the first threshold, it means that the confidence of the obtained first holding method is not high and it may be an unstable instantaneous state. At this time, the counter is increased to continuously obtain multiple first holding methods. If the number of the same type of first holding methods obtained is greater than or equal to the second threshold, it means that the first holding method of this type is maintained during the time period, that is, the first holding method of this type is in a continuous state. At this time, the first holding method of this type is determined as the second holding method; if the number of the same type of first holding method obtained is less than the second threshold, it means that the first holding method of this type is not in a continuous state. At this time, the first holding method of the type that appears the most times among the multiple first holding methods is determined as the second holding method, wherein the second threshold and the multiple first holding methods are both empirical values. For example, the multiple first holding methods are 10 first holding methods and the second threshold is 6.
[0074] For example, when the first threshold is 90% and the second threshold is 6, if the first holding method and its probability value are right-hand holding 94%, the second holding method is determined to be right-hand holding; if the first holding method and its probability value are right-hand holding 80%, the counter count is increased and the first holding method is output continuously for 10 times. When the number of times the first holding method is right-hand holding reaches 6 times, the second holding method is determined to be right-hand holding.
[0075] The second type is, if the first holding method is the same as the previous second holding method, the previous second holding method is determined to be the second holding method; if the first holding method is different from the previous second holding method, the second holding method is determined according to the first holding method obtained within the first preset time period.
[0076] The previous second holding method indicates the holding method of the terminal determined last time, which is a continuous state. The second method can be used simultaneously with the first method or alone. When the second method is used simultaneously with the first method, the first holding method is the second holding method obtained in the first method, which is a continuous state with stability; when the second method is used alone, the first holding method is the instantaneous state obtained at the current moment.
[0077] If the first holding method is the first output result, the first holding method is determined as the second holding method; if the first holding method is not the first output result, the second holding method is determined according to whether the first holding method is the same as the previous second holding method. If the first holding method is the same as the previous second holding method, it means that the previous second holding method is still in a continuous state, and the previous second holding method is determined as the current second holding method; if the first holding method is different from the previous second holding method, it means that the previous second holding method has changed, and it is necessary to determine whether the state change is an instantaneous state change or a continuous state change. If it is an instantaneous state change, it means that the previous second holding method will continue after the change, and the current second holding method is still the previous second holding method; if it is a continuous state change, it means that the changed state is a continuous state, and the current second holding method is the changed first holding method. All output first holding modes within a first preset time period are obtained, and based on all output first holding modes, it is determined whether the change in the previous second holding mode is an instantaneous state change or a continuous state change, so as to further determine the current second holding mode.
[0078] In one example, within a first preset time period, if the first holding method is the same as the previous second holding method, the previous second holding method is determined to be the second holding method; if the first holding method is not the same as the previous second holding method, the first holding method is determined to be the second holding method.
[0079] If the first holding method is the same as the previous second holding method, it means that the first holding method different from the previous second holding method is in a momentary state, and the previous second holding method has undergone a momentary state change and then changed back to the previous state. At this time, the previous second holding method is determined as the second holding method, and the first preset time period ends at this time; if the first holding method is not the same as the previous second holding method, it means that the first holding method different from the previous second holding method is in a continuous state, and the previous second holding method has undergone a continuous state change. At this time, the first holding method different from the previous second holding method is determined as the second holding method.
[0080] For example, the previous second holding method is left-hand holding, and the first preset time period is 10 hours. If the first holding method is left-hand holding, then the current second holding method is determined to be left-hand holding; if the first holding method is right-hand holding, then the timer is increased and the first holding method is continuously output. If the first holding method is left-hand holding when timer=6, then the current second holding method is determined to be left-hand holding, and the timer is reset to zero. If the first holding method is right-hand holding until timer=10, then the current second holding method is determined to be right-hand holding.
[0081] The third method is to obtain at least two first holding methods and probability values of the first holding method within a second preset time period, and determine the first holding method corresponding to the maximum value of the probability values as the second holding method.
[0082] The third method can be used simultaneously with the first method or alone. When the third method is used simultaneously with the first method, the first holding method is the second holding method obtained in the first method, which is a stable continuous state; when the third method is used alone, the first holding method is the instantaneous state obtained at the current moment. The second preset time period is an empirical value, for example, 10 seconds. The first holding method corresponding to the maximum value in the probability value within the second preset time period is the holding method with the greatest probability within the time period, and the first holding method with the greatest probability is determined as the second holding method.
[0083] For example, 100 first holding modes are obtained in 10 seconds, among which the probability value of right-hand holding is the largest, which is 86%, and then the second holding mode is right-hand holding.
[0084] In some possible implementations, the updating frequency of the holding way of the terminal is determined according to the application scenario of the holding way of the terminal.
[0085] The application scenarios of the terminal holding method include the following two situations:
[0086] The first one is that when it is applied to the anti-mistouch function, the determination of the holding method needs to be more sensitive, so the updating frequency of the holding method needs to be faster. The updating frequency of the holding method can be accelerated by increasing the number of holding method determinations and reducing the scenarios where no judgment is made. For example, when the average of the three-axis angular velocity is less than 0.02, the holding method is not judged. When the average of the three-axis angular velocity is less than 0.01, the holding method is not judged; a lighter neural network algorithm can also be used, such as reducing the number of input feature data, or modifying the convolution layer in the neural network algorithm structure to a MobileNet structure model, thereby reducing the amount of calculation and making the updating frequency of the holding method faster; a high-sensitivity and low-latency method can also be used when determining the second holding method, for example, lowering the first threshold, the second threshold, or shortening the first preset period, the second preset period, so that the updating frequency of the holding method is faster.
[0087] The second type, when applied to the auxiliary signal enhancement function, requires a more accurate determination result of the holding method, but the updating frequency of the holding method can be slower. More training data can be obtained by increasing the preset time length to make the accuracy of the trained neural network algorithm higher, so that the determination result of the holding method is more accurate; the number of holding method determinations can also be reduced, and the number of scenarios where no judgment is made can be increased. For example, when the mean of the three-axis angular velocity is less than 0.04 or greater than 0.5, the holding method is not judged, and the previous determination result is maintained to make the determination result of the holding method more accurate; more complex neural network algorithms can also be used, or the types of output results can be increased to make the determination result of the holding method more accurate; low sensitivity and high latency methods can also be used when determining the second holding method, for example, the first threshold, the second threshold, or the first preset period, the second preset period can be lengthened to obtain a high-confidence holding method.
[0088] It should be noted that the auxiliary signal enhancement function includes determining the use of the terminal's transmitting or receiving antenna according to the holding method. The part of the terminal device held by the left or right hand usually has many signal receiving and transmitting antennas. If the antenna is covered by the hand, the signal of the terminal device will become worse. At this time, it is necessary to use the antenna in the terminal device that is not covered by the hand to receive and transmit signals. Therefore, when applied to the auxiliary signal enhancement function, the holding method determination result needs to be more accurate.
[0089] In an exemplary embodiment of the present disclosure, inertial sensor data is obtained and characteristic data of the inertial sensor is calculated, and a first holding method of the terminal and a probability value of the first holding method are obtained according to the inertial sensor data and / or the characteristic data, and a second holding method is obtained according to the first holding method and the probability value of the first holding method, and the second holding method is determined as the holding method of the terminal, so that the obtained holding method of the terminal has a high confidence level, so as to optimize the extended functions related to the holding method in the terminal, such as optimizing the anti-mistouch function and the auxiliary signal enhancement function of the terminal, and in the present disclosure, only the relevant data under the terminal usage status is obtained through the inertial sensor, without relying on other hardware structures, and has the characteristics of low power consumption and wide applicability.
[0090] In an exemplary embodiment of the present disclosure, a device for determining a holding method is provided, which is applied to a terminal. Figure 5 is a block diagram of a device for determining a holding mode according to an exemplary embodiment. Figure 5 As shown, the device comprises:
[0091] The acquisition module 501 is configured to acquire inertial sensor data and / or feature data within a preset time period, wherein the inertial sensor data includes at least one of an inertial sensor reading and a statistical value of the inertial sensor reading, and the feature data includes at least one of time domain feature data, frequency domain feature data, and spectrum feature data of the inertial sensor data;
[0092] The calculation module 502 is configured to obtain a first holding mode of the terminal and a probability value of the first holding mode according to the inertial sensor data and / or the feature data, wherein the first holding mode includes at least one of the following: left-hand holding, right-hand holding, and both-hand holding;
[0093] The determination module 503 is configured to determine a second holding way according to the first holding way and the probability value of the first holding way, and determine the second holding way as the holding way of the terminal.
[0094] In an exemplary embodiment, the calculation module 502 is further configured to:
[0095] determining a neural network algorithm based on at least one of a memory size of the terminal, a placement of the terminal, and a position of the inertial sensor in the terminal;
[0096] The inertial sensor data and / or the feature data are input into the neural network algorithm to obtain the first holding manner and the probability value of the first holding manner.
[0097] In an exemplary embodiment, the determining module 503 is further configured to:
[0098] If the probability value of the first holding way is greater than or equal to a first threshold, determining that the first holding way is the second holding way;
[0099] If the probability value of the first holding method is less than the first threshold, and the number of acquisitions of the same type of the first holding method is greater than or equal to a second threshold, it is determined that the first holding method is the second holding method.
[0100] In an exemplary embodiment, the determining module 503 is further configured to:
[0101] If the first holding manner is the same as the previous second holding manner, determining the previous second holding manner to be the second holding manner;
[0102] If the first holding manner is different from the previous second holding manner, the second holding manner is determined according to the first holding manner obtained within a first preset time period.
[0103] In an exemplary embodiment, the determining module 503 is further configured to:
[0104] In the first preset time period, if the first holding manner is the same as the previous second holding manner, determining the previous second holding manner to be the second holding manner;
[0105] If the first holding way is not the same as the previous second holding way, it is determined that the first holding way is the second holding way.
[0106] In an exemplary embodiment, the determining module 503 is further configured to:
[0107] At least two probability values of the first holding manner and the first holding manner are obtained within a second preset time period, and the first holding manner corresponding to the maximum value of the probability values is determined as the second holding manner.
[0108] In an exemplary embodiment, the determining module 503 is further configured to:
[0109] The updating frequency of the holding way of the terminal is determined according to the application scenario of the holding way of the terminal.
[0110] In an exemplary embodiment, the inertial sensor includes an acceleration sensor and an angular velocity sensor.
[0111] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.
[0112] Figure 6is a block diagram of a terminal device 600 for executing a method for determining a holding manner according to an exemplary embodiment.
[0113] Reference Figure 6 The terminal device 600 may include one or more of the following components: a processing component 602 , a memory 604 , a power component 606 , a multimedia component 608 , an audio component 610 , an input / output (I / O) interface 612 , a sensor component 614 , and a communication component 616 .
[0114] The processing component 602 generally controls the overall operation of the terminal device 600, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 608 and the processing component 602.
[0115] The memory 604 is configured to store various types of data to support operations on the terminal device 600. Examples of such data include instructions for any application or method operating on the terminal device 600, contact data, phone book data, messages, pictures, videos, etc. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.
[0116] The power supply component 606 provides power to various components of the terminal device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the terminal device 600.
[0117] The multimedia component 608 includes a screen that provides an output interface between the terminal device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 608 includes a front camera and / or a rear camera. When the terminal device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.
[0118] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC), and when the terminal device 600 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 604 or sent via the communication component 616. In some embodiments, the audio component 610 also includes a speaker for outputting audio signals.
[0119] I / O interface 612 provides an interface between processing component 602 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.
[0120] The sensor assembly 614 includes one or more sensors for providing various aspects of status assessment for the terminal device 600. For example, the sensor assembly 614 can detect the open / closed state of the terminal device 600, the relative positioning of the components, such as the display and keypad of the terminal device 600, and the sensor assembly 614 can also detect the position change of the terminal device 600 or a component of the terminal device 600, the presence or absence of contact between the user and the terminal device 600, the orientation or acceleration / deceleration of the terminal device 600, and the temperature change of the terminal device 600. The sensor assembly 614 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 614 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0121] The communication component 616 is configured to facilitate the communication between the terminal device 600 and other devices in a wired or wireless manner. The terminal device 600 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0122] In an exemplary embodiment, the terminal device 600 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above method.
[0123] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the instructions can be executed by a processor 620 of a terminal device 600 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0124] A non-temporary computer-readable storage medium, when the instructions in the storage medium are executed by a processor of a terminal device, enables the terminal device to execute a method for determining a holding manner, the method including any of the above methods.
[0125] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed in this disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.
[0126] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.
Claims
1. A method for determining a holding method, characterized in that: The method comprises: Acquire inertial sensor data and / or feature data within a preset time period, wherein the inertial sensor data includes at least one of an inertial sensor reading and a statistical value of the inertial sensor reading, and the feature data includes at least one of time domain feature data, frequency domain feature data, and spectrum feature data of the inertial sensor data; Obtaining, according to the inertial sensor data and / or the characteristic data, a first holding mode of the terminal and a probability value of the first holding mode, wherein the first holding mode includes at least one of the following: left-hand holding, right-hand holding, and both-hand holding; A second holding way is determined according to the first holding way and a probability value of the first holding way, and the second holding way is determined as the holding way of the terminal.
2. The method for determining the holding mode according to claim 1, characterized in that: The obtaining, according to the inertial sensor data and / or the feature data, a first holding manner of the terminal and a probability value of the first holding manner includes: determining a neural network algorithm based on at least one of a memory size of the terminal, a placement of the terminal, and a position of the inertial sensor in the terminal; The inertial sensor data and / or the feature data are input into the neural network algorithm to obtain the first holding manner and the probability value of the first holding manner.
3. The method for determining the holding mode according to claim 1, characterized in that: The determining the second holding manner according to the first holding manner and the probability value of the first holding manner includes: If the probability value of the first holding way is greater than or equal to a first threshold, determining that the first holding way is the second holding way; If the probability value of the first holding method is less than the first threshold, and the number of acquisitions of the same type of the first holding method is greater than or equal to a second threshold, the first holding method is determined to be the second holding method.
4. The method for determining the holding mode according to claim 1, characterized in that: The determining the second holding manner according to the first holding manner and the probability value of the first holding manner includes: If the first holding manner is the same as the previous second holding manner, determining the previous second holding manner to be the second holding manner; If the first holding manner is different from the previous second holding manner, the second holding manner is determined according to the first holding manner obtained within a first preset time period.
5. The method for determining the holding mode according to claim 4, characterized in that: The determining the second holding manner according to the first holding manner obtained within a first preset time period includes: In the first preset time period, if the first holding manner is the same as the previous second holding manner, determining the previous second holding manner to be the second holding manner; If the first holding way is not the same as the previous second holding way, it is determined that the first holding way is the second holding way.
6. The method for determining the holding mode according to claim 1, characterized in that: The determining the second holding manner according to the first holding manner and the probability value of the first holding manner includes: At least two probability values of the first holding manner and the first holding manner are obtained within a second preset time period, and the first holding manner corresponding to the maximum value of the probability values is determined as the second holding manner.
7. The method for determining the holding mode according to claim 1, characterized in that: The method further comprises: The updating frequency of the holding way of the terminal is determined according to the application scenario of the holding way of the terminal.
8. The method for determining the holding mode according to claim 1, characterized in that: The inertial sensor includes an acceleration sensor and an angular velocity sensor.
9. A device for determining a holding method, characterized in that: The device comprises: an acquisition module, configured to acquire inertial sensor data and / or feature data within a preset time period, wherein the inertial sensor data includes at least one of an inertial sensor reading and a statistical value of the inertial sensor reading, and the feature data includes at least one of time domain feature data, frequency domain feature data, and spectrum feature data of the inertial sensor data; a calculation module, configured to obtain a first holding mode of the terminal and a probability value of the first holding mode according to the inertial sensor data and / or the feature data, wherein the first holding mode includes at least one of the following: left-hand holding, right-hand holding, and both-hand holding; The determination module is configured to determine a second holding method according to the first holding method and a probability value of the first holding method, and determine the second holding method as the holding method of the terminal.
10. A terminal device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute the method as claimed in any one of claims 1 to 8.
11. A non-transitory computer-readable storage medium, characterized in that: When the instructions in the storage medium are executed by a processor of a terminal device, the terminal device is enabled to execute the method as described in any one of claims 1 to 8.