Touch control method, device and computer readable storage medium

Abstract

The present disclosure relates to a touch method, device and computer readable storage medium, wherein the touch method is applied to a mobile terminal, comprising: acquiring sensing data collected by at least two inertial sensors in the mobile terminal, wherein the at least two inertial sensors are used to generate the sensing data according to a touch action on the mobile terminal; generating touch coordinates of the touch action based on the sensing data; and responding to the touch action based on the touch coordinates. In this way, since the touch coordinates of the touch action are determined, the further processing based on the touch coordinates can reduce the false touch rate of the touch action acting on the mobile terminal, and improve the user experience.

Description

Technical Field

[0001] This disclosure relates to the field of electronic device technology, and in particular to a touch control method, apparatus and computer-readable storage medium. Background Technology

[0002] As mobile phones integrate increasingly diverse functions, they can perform a wide range of additional features. For example, when holding a phone, users can perform specific shortcut gestures by tapping or swiping the back or side of the screen. However, due to the complexity of everyday scenarios, many situations can generate signals similar to tapping or swiping the back or side of the screen, leading to incorrect interpretations of shortcut gestures. In some cases, even without actual touch on the back or side of the screen, the phone might mistakenly detect a corresponding shortcut gesture and activate unnecessary functions, causing inconvenience to the user. Summary of the Invention

[0003] This disclosure provides a touch control method, apparatus, and computer-readable storage medium.

[0004] According to a first aspect of the present disclosure, a touch control method is provided, applied to a mobile terminal, comprising:

[0005] Acquire sensing data collected by at least two inertial sensors within a mobile terminal, wherein the at least two inertial sensors are used to generate the sensing data based on touch actions on the mobile terminal;

[0006] Based on the sensor data, the touch coordinates of the touch action are generated;

[0007] The touch action is responded to based on the touch coordinates.

[0008] Optionally, responding to the touch action based on the touch coordinates includes:

[0009] When the touch coordinates meet the preset position triggering conditions, the touch action is responded to.

[0010] Optionally, responding to the touch action when the touch coordinates satisfy a preset position trigger condition includes:

[0011] The touch action is responded to when the current area of ​​the touch coordinates matches the preset active area.

[0012] Optionally, the method includes:

[0013] Obtain the preset sub-area corresponding to each type of touch action; where different types of touch actions correspond to different preset sub-areas.

[0014] The step of responding to the touch action when the current area of ​​the touch coordinates matches the preset effective area includes:

[0015] When the area where the touch coordinates are currently located matches the preset sub-area corresponding to the touch action, the touch action is responded to.

[0016] Optionally, generating the touch coordinates of the touch action based on the sensing data includes:

[0017] The sensor data is input into a preset target prediction model to obtain the touch coordinates of the touch action.

[0018] Optionally, the method further includes:

[0019] The historical sample data collected by the mobile terminal is obtained, wherein the historical sample data includes: historical sensor data and historical touch coordinates corresponding to the historical sensor data;

[0020] The historical sample data is input into the initial prediction model to be trained for iterative processing until the difference between the output touch coordinates and the historical touch coordinates in the historical sample data satisfies the convergence condition, thus obtaining the target prediction model.

[0021] Optionally, acquiring sensing data from at least two inertial sensors within the mobile terminal includes:

[0022] Sensing data is acquired based on a first inertial sensor located in a first region and a second inertial sensor located in a second region on the back cover of the mobile terminal; wherein the first region and the second region are distributed on both sides of an axis parallel to the short side direction of the mobile terminal.

[0023] According to a second aspect of the present disclosure, a touch device is provided, applied to a mobile terminal, comprising:

[0024] A data acquisition module is used to acquire sensing data collected by at least two inertial sensors in a mobile terminal, wherein the at least two inertial sensors are used to generate the sensing data based on touch actions on the mobile terminal.

[0025] A coordinate generation module is used to generate touch coordinates for the touch action based on the sensor data;

[0026] A response module is used to respond to the touch action based on the touch coordinates.

[0027] Optionally, the response module is further configured to respond to the touch action when the touch coordinates meet a preset position triggering condition.

[0028] Optionally, the response module is further configured to respond to the touch action when the current area of ​​the touch coordinates matches a preset active area.

[0029] Optionally, the device further includes:

[0030] The region acquisition module is used to acquire the preset sub-operating regions corresponding to various types of touch actions; different types of touch actions correspond to different preset sub-operating regions.

[0031] The response module is also used for:

[0032] When the area where the touch coordinates are currently located matches the preset sub-area corresponding to the touch action, the touch action is responded to.

[0033] Optionally, the data acquisition module is further configured to acquire sensing data based on a first inertial sensor located in a first region and a second inertial sensor located in a second region on the back cover of the mobile terminal; wherein the first region and the second region are distributed on both sides of an axis parallel to the short side direction of the mobile terminal.

[0034] According to a third aspect of the present disclosure, a touch device is provided, comprising:

[0035] processor;

[0036] Memory used to store processor-executable instructions;

[0037] The processor is configured to, when executing executable instructions stored in the memory, implement the method described in any of the first aspects above.

[0038] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium is provided, wherein computer-executable instructions are stored therein, which, when executed by a processor, implement the steps of the method provided in any of the first aspects described above.

[0039] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0040] The touch control method provided in this disclosure, after acquiring at least two sensor data points generated by touch actions on a mobile terminal, generates touch coordinates for the touch actions based on the sensor data, and then responds to the touch actions based on the touch coordinates. In this way, on the one hand, touch action responses can be executed based on detected sensor data, enriching the application scenarios of function responses on mobile terminals and providing diverse touch experiences. On the other hand, touch coordinates are introduced into the function response, which can improve the accuracy of touch action responses; for example, the touch coordinates of the touch operation can be compared with coordinates within a preset effective area, and the corresponding function will only be responded to when the coordinates match successfully, thereby improving the accuracy of touch action responses.

[0041] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0042] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0043] Figure 1 This is a flowchart illustrating a touch method according to an exemplary embodiment.

[0044] Figure 2 This is a schematic diagram illustrating a mobile terminal comprising two inertial sensors according to an exemplary embodiment.

[0045] Figure 3 This is a flowchart illustrating a touch method according to an exemplary embodiment.

[0046] Figure 4 This is a schematic diagram illustrating the determination of touch coordinates according to an exemplary embodiment.

[0047] Figure 5 This is a schematic diagram of the structure of a touch device according to an exemplary embodiment.

[0048] Figure 6 This is a block diagram illustrating a touch device according to an exemplary embodiment. Detailed Implementation

[0049] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0050] This disclosure provides a touch control method. Figure 1 This is a flowchart illustrating a touch method according to an exemplary embodiment, such as... Figure 1 As shown, this touch method is applied to a mobile terminal and includes the following steps:

[0051] Step 101: Acquire sensing data collected by at least two inertial sensors in the mobile terminal, wherein the at least two inertial sensors are used to generate the sensing data based on touch actions on the mobile terminal;

[0052] Step 102: Based on the sensor data, generate the touch coordinates of the touch action;

[0053] Step 103: Respond to the touch action based on the touch coordinates.

[0054] It should be noted that this touch control method can be applied to any mobile terminal, such as a smartphone, tablet, or wearable electronic device.

[0055] The mobile terminal includes a housing, a display screen, and at least two inertial measurement units (IMUs) located inside the housing.

[0056] The housing includes a back cover, which is a portion of the housing that faces away from the display screen. For example, tapping or vibrating the back cover or the display screen causes changes in data collected by an inertial sensor inside the housing; by monitoring these changes, the type of touch action can be determined.

[0057] In this disclosure, the touch action applied to the mobile terminal may include: a touch action applied to the mobile terminal display screen, or a touch action applied to the back cover of the mobile terminal.

[0058] For example, the touch actions performed on the mobile terminal can include operations such as single-clicking or double-clicking that cause changes in the data detected by the inertial sensor; different touch actions correspond to different functions. By distinguishing the sensor data, the touch action can be determined, and then the corresponding function can be responded to. For example, a double-click operation on the display screen can turn on the display screen, a double-click operation on the back cover can turn on the screenshot function, and a single-click operation on the back cover can turn on the flashlight function.

[0059] In some embodiments, when a user is holding and using a mobile terminal, and an application is already running on the screen, if they want to perform some quick gestures without affecting the application's execution, they can trigger the corresponding quick gesture by performing the touch action on the back cover of the mobile terminal. For example, when watching a video, double-tapping the back cover of the mobile terminal can be used to take a screenshot of the video screen.

[0060] Thus, the acquisition of sensing data collected by at least two inertial sensors within the mobile terminal in step 101 may include:

[0061] Acquire sensor data collected by at least two inertial sensors within the mobile terminal, which are collected by touch actions performed on the back cover of the mobile terminal.

[0062] It should be noted that when the back cover of a mobile terminal is tapped, the mobile terminal will rotate slightly around its axis. Therefore, the signal detected by the inertial sensor located inside the mobile terminal will show a sinusoidal characteristic or a peak-like characteristic; and the moment this peak appears is the moment of the tap.

[0063] In this embodiment, a three-axis coordinate system (xyz) can be established based on the back cover surface of the mobile terminal and the thickness direction of the mobile terminal. Based on the three-axis coordinate system, when a signal resembling a sine wave appears on one of the detected three-axis data axes, it is considered that a tap operation has been detected. For example, when the mobile terminal is held in portrait mode, a sine wave signal may be detected on the x-axis.

[0064] To ensure detection accuracy, the mobile terminal in this embodiment includes at least two inertial sensors. Furthermore, the at least two inertial sensors are located at different positions within the mobile terminal (e.g., Figure 2 (As shown). When sensing data is obtained based on the at least two inertial sensors, a signal resembling a sine wave will appear on the same axis (e.g., the x-axis) in the data detected by each inertial sensor. However, due to the different distances between the impact position and each inertial sensor, the peak value of the sine wave detected by each inertial sensor will be different. The peak value of the sine wave detected by the inertial sensor closer to the impact position will be larger.

[0065] Here, when the mobile terminal is placed vertically perpendicular to the ground, if a tap is made on the back cover or display screen that corresponds to the location of the inertial sensor inside the mobile terminal, assuming a signal similar to a sine wave will appear on the x-axis, then in this case of direct alignment, the signals on other axes may not fluctuate, and the amount of signal change will be 0.

[0066] When the tap is not made on the back cover or display screen that corresponds directly to the location of the inertial sensor in the mobile terminal (i.e., there is a deviation), both the y-axis and z-axis signals will fluctuate. In this case, the changes in the y-axis and z-axis signals will not be zero.

[0067] After obtaining the data detected by each inertial sensor, the data detected by each inertial sensor (e.g., two inertial sensors) can be combined to obtain the sensing data of this tap.

[0068] Furthermore, acquiring the sensing data of the touch operation based on data detected from at least two inertial sensors may include:

[0069] The sensor data for touch operation are obtained by averaging the data detected on the corresponding coordinate axes from the at least two inertial sensors.

[0070] In this way, the averaging method can combine the position of the touch action and the positions of at least two inertial sensors to obtain sensing data that is more consistent with the actual touch situation, thereby improving the accuracy of the sensing data and facilitating the accurate determination of the touch coordinates in the future.

[0071] In this way, some shortcut gestures can be implemented without affecting the execution of the application; compared with shortcut gestures triggered by touch actions on the display screen, this is more conducive to the implementation of the touch method in scenarios where the display screen is being used.

[0072] The inertial sensor is used to detect the movement of the mobile terminal, such as rotation or acceleration. When there is a touch action on the back cover of the mobile terminal, the inertial sensor inside the mobile terminal will generate sensing data based on the touch action.

[0073] The sensing data collected by the inertial sensor includes: sensing data corresponding to a single click or a double click. In some embodiments, the inertial sensor includes sensors that measure inertial forces, such as accelerometers and / or gyroscopes, rather than devices that detect touch operations, such as touchscreens or touch panels.

[0074] In this embodiment of the disclosure, the inertial sensor can also be reused for motion state detection of a mobile terminal.

[0075] It should be noted that, in order to more accurately determine the current touch action based on the detected sensor data, the mobile terminal in this embodiment includes at least two inertial sensors. Thus, when the at least two sensors are located in different positions, the touch action can be determined more accurately based on the cooperation between the collected sensor data. Taking a mobile terminal containing two inertial sensors as an example, in some embodiments, acquiring the sensor data collected by the at least two inertial sensors in the mobile terminal includes:

[0076] Sensing data is acquired based on a first inertial sensor located in a first region and a second inertial sensor located in a second region on the back cover of the mobile terminal; wherein the first region and the second region are distributed on both sides of an axis parallel to the short side direction of the mobile terminal.

[0077] In some embodiments, the first region may be the region where the upper half of the back cover of the mobile terminal is located, and the upper half of the back cover of the mobile terminal may be the part where the camera is located.

[0078] The second region may be the region where the lower half of the mobile terminal back cover is located; the lower half of the mobile terminal back cover may be the region other than the first region among the two regions on the mobile terminal back cover divided by an axis parallel to the short side direction.

[0079] Here, as Figure 2 As shown, Figure 2 This is a schematic diagram illustrating a mobile terminal including two inertial sensors according to an exemplary embodiment. When the mobile terminal includes two inertial sensors, the first and second inertial sensors can be respectively disposed at different positions on the back cover, thus expanding the detection coverage area. Furthermore, to cover an even larger detection range, the two inertial sensors can be disposed at relatively distant positions on the back cover. For example, as... Figure 2 As shown, the first inertial sensor 201 is located in the upper half of the back cover, that is, the first inertial sensor 201 is located in the first area, while the second inertial sensor 202 is located in the lower half of the back cover, that is, the second inertial sensor 202 is located in the second area. In this way, since the two inertial sensors are far apart in position, a larger detection range can be covered.

[0080] It's important to note that in related technologies, when a user taps the lower half of the back or the edge of a mobile terminal, the signal detection might be very similar to that of tapping the lower half of the front or the upper half of the back. This can lead the sensor data to mistakenly assume the user tapped the upper half of the back (when in reality, the lower half of the front might have been tapped). Therefore, the current touch action is determined based on the collected sensor data. If the tap matches the corresponding type of touch action or a shortcut gesture, the corresponding function is triggered. However, the tap might actually be an unconscious touch, fulfilling a shortcut gesture but not triggering the desired function. In this embodiment, in addition to considering the pairing of sensor data and corresponding shortcut gestures, the location of the touch action is also considered. Further judgment of the location of the touch action is required, and the function corresponding to the touch action is only triggered when the location also meets the specified criteria.

[0081] In some embodiments, responding to the touch action based on the touch coordinates includes:

[0082] When the touch coordinates meet the preset position triggering conditions, the touch action is responded to.

[0083] Here, the location triggering condition can be: the condition that the touch coordinates of the touch action need to be in a preset effective area.

[0084] The preset operating area can be determined based on the user's operating habits or ease of use. For example, when holding a mobile terminal, the upper half of the back cover is more convenient for touch control. However, for quick touch responses on the display screen, since application icons are likely more concentrated on the upper half of the screen, touch on the upper half may result in opening an application rather than a quick function. Therefore, the lower half of the display screen is more convenient for touch control.

[0085] Thus, for the quick touch response on the display side, the preset operating area can be the lower half of the display (i.e., the part opposite the front camera). For the quick touch response on the back cover side, the preset operating area can be the upper half of the back cover (i.e., the part where the rear camera is located).

[0086] Based on the position triggering condition, in some embodiments, responding to the touch action when the touch coordinates satisfy the preset position triggering condition includes:

[0087] The touch action is responded to when the current area of ​​the touch coordinates matches the preset active area.

[0088] Here, the touch action will only be responded to when the current area of ​​the touch coordinates matches the preset effective area. This further positional matching can minimize accidental touches. It can be understood that a match between the current area of ​​the touch coordinates and the preset effective area indicates that the touch coordinates are within the preset effective area. Furthermore, when the current area of ​​the touch coordinates does not match the preset effective area, i.e., when the current area of ​​the touch coordinates is not within the preset effective area, the touch action will not be responded to.

[0089] The preset active area is the area that can trigger the touch function of the back cover.

[0090] For example, assuming the preset effective area is the upper half of the back cover of the mobile terminal, the touch coordinates corresponding to the detected touch action also need to be located within the upper half of the back cover of the mobile terminal. Thus, after determining the specific touch action based on the acquired sensor data, the position of the touch action is further determined. Only when the position is also within the preset effective area will the function corresponding to the touch action be responded to; if the position of the touch action is not within the preset effective area, the touch action will not be responded to, that is, the function corresponding to the touch action will not be implemented.

[0091] Taking a touch scenario on the back cover as an example, in this embodiment of the disclosure, assuming that the focus is on the quick touch response on the back, and the preset operating area of ​​the quick touch response on the back is in the upper half of the back, then this disclosure addresses the following: even if the sensor data detected by a touch action on the bezel or display screen is similar to or the same as the sensor data detected by a touch action on the back cover, the function corresponding to the touch action on the bezel or display screen will not be responded to. Instead, after determining the specific touch action based on the sensor data detected by the inertial sensor, the touch coordinates of the touch action are further detected. Only when the touch coordinates of the touch action are located in the upper half of the back will the touch action be responded to.

[0092] As a specific example, taking a touch scenario on the back cover as an example, suppose a double-click operation on the back cover can activate the screenshot function, and suppose the preset position trigger condition is that the touch coordinates of the touch action are located in the upper half of the back cover. If the sensor data currently detected on the mobile terminal indicates that the touch action is a double-click, and a double-click corresponds to activating the screenshot function. Since this embodiment also considers the touch coordinates of the touch action (if the touch coordinates were not considered, the screenshot would be directly executed based on the detected double-click operation), the touch coordinates (or specific position) of the double-click operation on the back cover will be further determined. The double-click operation will only be responded to when the touch coordinates of the double-click operation meet the preset position trigger condition (i.e., the touch coordinates are located in the upper half of the back cover). Conversely, if the touch coordinates of the double-click operation do not meet the preset position trigger condition, the double-click operation will not be responded to even if the sensor data indicates that the touch action is a double-click.

[0093] In this way, since the function is responded to based on the touch coordinates of the touch action, the response of the function corresponding to the touch action has an additional trigger condition, which improves the accuracy of the quick touch response.

[0094] In some embodiments, position matching can be achieved by setting the same preset effective area for different touch actions; however, different functions can be triggered based on different touch actions. For example, assuming the upper half of the back cover is the preset effective area, the corresponding function can only be responded to when a touch action is performed on the upper half of the back cover. Therefore, responding to the touch action when the current area of ​​the touch coordinates matches the preset effective area can be achieved by responding to the touch action when the current area of ​​the touch coordinates is located on the upper half of the back cover.

[0095] In some embodiments, position matching can also involve setting different preset active areas for different touch actions. For example, performing a double-click operation in a first sub-area on the back cover triggers the flashlight function; performing a double-click operation in a second sub-area on the back cover triggers the voice interaction function; the first sub-area is different from the second sub-area. That is, the method further includes:

[0096] Obtain the preset sub-area corresponding to each type of touch action; where different types of touch actions correspond to different preset sub-areas.

[0097] The above-mentioned response to the touch action when the current area of ​​the touch coordinates matches the preset effective area includes:

[0098] When the area where the touch coordinates are currently located matches the preset sub-area corresponding to the touch action, the touch action is responded to.

[0099] In this embodiment of the disclosure, the various types of touch actions can be: single-click type touch actions or double-click type touch actions.

[0100] Before executing a touch shortcut response, the preset sub-area corresponding to each type of touch action is first obtained. Then, the current area of ​​the touch coordinates is matched with the preset sub-area corresponding to each type of touch action. Based on the matching result, it is determined whether to respond to the touch action.

[0101] For example, when a double-click operation is performed in the first sub-region on the back cover to trigger the flashlight function, the preset sub-area corresponding to the double-click type touch action is the first sub-region. Therefore, the double-click operation is only responded to and the flashlight function is activated when the current area of ​​the double-click operation is the first sub-region; when the current area of ​​the double-click operation is not the first sub-region, the double-click operation is not responded to and the flashlight function is not activated.

[0102] Here, preset sub-areas can be assigned to various types of touch actions based on the frequency of use of shortcut functions. For example, if the screenshot function is used frequently and corresponds to a double-tap type touch action, the preset sub-area for the double-tap type touch action can be set in the middle area of ​​the upper half of the back cover. Thus, after determining the current location of the touch coordinates for a double-tap type touch action, this current location is matched with the middle area of ​​the upper half of the back cover; only when a match is successful will the double-tap type touch action be responded to.

[0103] Therefore, in this embodiment, it is necessary to match the area where the touch coordinates are located with the preset sub-operation area corresponding to the touch action. Only when the match is successful can it be determined that the touch action should be responded to and the corresponding function should be implemented. This can minimize accidental touches as much as possible.

[0104] Here, since it is necessary to match the current area of ​​the touch coordinates with the preset sub-areas corresponding to each type of touch action, before matching, it is necessary to determine the specific type of touch action (single-click type touch action or double-click type touch action) based on the currently detected sensor data.

[0105] Thus, in some embodiments, the method further includes:

[0106] Obtain the mapping relationship between preset sensor data and the type of touch action;

[0107] The type of touch action is determined based on the sensor data and the mapping relationship.

[0108] In this embodiment of the disclosure, a mapping relationship between sensing data and the type of touch action can be pre-stored; the mapping relationship indicates the pairing relationship between sensing data and the type of touch action.

[0109] The mapping relationship can be collected during the historical use of the mobile terminal. For example, during historical use, a single click operation is performed at multiple locations within a preset area on the back cover of the mobile terminal to obtain multiple corresponding sensor data. Then, any sensor data in the set of multiple sensor data can be considered as the sensor data corresponding to a single click-type touch action.

[0110] Here, the number of times the same or similar sensor data is detected within a preset time period can be used to determine whether the current touch action is a double tap or a single tap. For example, if the same or similar sensor data is detected twice within 0.1 seconds, the current touch action is considered a double tap.

[0111] Furthermore, generating the touch coordinates of the touch action based on the sensor data can be achieved based on a preset target prediction model:

[0112] In some embodiments, step 102, generating the touch coordinates of the touch action based on the sensing data, may include:

[0113] Step 1021: Input the sensing data into the preset target prediction model to obtain the touch coordinates of the touch action.

[0114] In this embodiment, the touch coordinates can be coordinates in a coordinate system established on the surface of the back cover of the mobile terminal. For example, the touch coordinates can be (x, y) or (x, y, z), where the x-axis can be the axis containing the short side of the mobile terminal, the y-axis can be the axis containing the long side of the mobile terminal, and the z-axis can be the axis along the thickness direction of the mobile terminal. Here, when the inertial sensor is a sensor that detects along two axes, the obtained touch coordinates have only two parameters, such as (x, y) as described above. When the inertial sensor is a sensor that detects along three axes, the obtained touch coordinates have three parameters, such as (x, y, z) as described above.

[0115] The mobile terminal has a pre-set target prediction model. When a touch action is detected on the back cover of the mobile terminal, the sensor data is acquired, and then the sensor data is processed based on the pre-set target prediction model to output the touch coordinates.

[0116] The target prediction model can be trained using historical sensor data and historical touch coordinates. The sensor data is input into the target prediction model, and the touch coordinates are output.

[0117] In some embodiments, Figure 3 This is a flowchart illustrating a touch method according to an exemplary embodiment. Figure 3 As shown, the establishment of the target prediction model can be as follows:

[0118] Step 301: Obtain historical sample data collected during the historical use of the mobile terminal, wherein the historical sample data includes: historical sensor data and historical touch coordinates corresponding to the historical sensor data;

[0119] Step 302: Input the historical sample data into the initial prediction model to be trained for iterative processing until the difference between the output touch coordinates and the historical touch coordinates in the historical sample data satisfies the convergence condition, and obtain the target prediction model.

[0120] It should be noted that steps 301 to 302 occur before step 1021. After obtaining the target prediction model, the sensor data acquired in step 101 can be processed based on the target prediction model to obtain the touch coordinates of the touch action.

[0121] It should be noted that the historical sensor data and historical touch coordinates obtained in a single collection are a set of corresponding data.

[0122] The initial prediction model to be trained can be any neural network model capable of making predictions. For example, a feedforward neural network (BP) model or a long short-term memory (LSTM) model. By training the initial prediction model with historical sample data collected during the historical use of the mobile terminal to continuously optimize the model's parameters, a target prediction model capable of accurately predicting touch coordinates can be obtained.

[0123] Since the historical sensor data and the corresponding historical touch coordinates acquired in a single acquisition are used as a set of input data in this embodiment, the amount of data processing is relatively small. Considering the speed of model training, a three-layer fully connected neural network model can also be selected as the initial prediction model to be trained in this embodiment. This disclosure does not limit the selection of the initial prediction model.

[0124] Here, the historical sample data includes: historical sensor data and historical touch coordinates corresponding to the historical sensor data. The historical sample data can be laboratory data, that is, data obtained by collecting sensor data and corresponding touch coordinates through multiple experiments before predicting touch coordinates.

[0125] The historical sensing data refers to the historical data detected by at least two inertial sensors placed inside the mobile terminal during its use. The historical touch coordinates refer to the coordinates of the corresponding positions of the historical sensing data detected when a touch action occurs during the use of the mobile terminal.

[0126] Before inputting the historical sensing data into the initial prediction model to be trained for iterative processing, it is necessary to remove erroneous data from the historical sample data.

[0127] The process of clearing erroneous data from historical sample data can be as follows: clearing the sensor data corresponding to a touch action when the touch coordinates and response function of the touch action do not match.

[0128] In other words, because the signals from tapping the upper half of the back of a phone are very similar to those from tapping the lower half of the front, in actual use, it's possible that within a preset time period, one tap is on the lower half of the front and another on the upper half of the back. However, based on the detected sensor data, it might be mistaken for two taps on the upper half of the back within the preset time period, thus triggering the double-tap convenience function. Therefore, the sensor data corresponding to the tap being on the lower half of the front and the upper half of the back is incorrect and cannot be used as data to trigger the double-tap convenience function.

[0129] Therefore, in this embodiment of the disclosure, it is necessary to clear such historical sample data and use the correctly matched sensor data and touch coordinates as a set of historical sample data to perform model training.

[0130] Here, after the acquired sensor data is input into the initial prediction model, if the output touch coordinates are extremely close to, or identical to, the historical touch coordinates in the historical sample data, then the prediction model at this time is considered the target prediction model. However, the situation where the output touch coordinates are identical to the historical touch coordinates in the historical sample data is extremely time-consuming and rarely occurs. Therefore, in this embodiment of the disclosure, when the output touch coordinates are extremely close to the historical touch coordinates in the historical sample data, the target prediction model is considered to have been obtained.

[0131] Considering the complexity of model training and experimental efficiency, the extreme values ​​of the output touch coordinates can be determined by setting convergence conditions to see if they are extremely close to the historical touch coordinates in the historical sample data.

[0132] In some embodiments, the convergence condition may be that the difference between the output touch coordinates and the historical touch coordinates satisfies a preset value. That is, a preset value is first set, and the difference between the output touch coordinates and the historical touch coordinates in the historical sample data is compared with the preset value. If the difference is less than the preset value, the convergence condition is considered to be satisfied, and the model parameters at this time are the model parameters of the final target prediction model, and the target prediction model is considered to have been obtained.

[0133] Here, if the difference between the output touch coordinates and the historical touch coordinates in the corresponding historical sample data is always not less than the preset value, the model parameters are continuously adjusted. After adjusting the model parameters, the new neural network model processes the historical sensor data, outputs the touch coordinates again, and then compares the touch coordinates with the historical touch coordinates in the corresponding historical sample data until the difference is less than the preset value, thus obtaining the target prediction model.

[0134] In other embodiments, the convergence condition may also be: the difference between the output touch coordinates and the corresponding historical touch coordinates in the historical sample data is minimized within a preset number of iterations. That is, the number of iterations is first set, and after a preset number of iterations, the model with the smallest difference between the output touch coordinates and the corresponding historical touch coordinates in the historical sample data is set as the target prediction model.

[0135] In other embodiments, a loss function can be set during model training to evaluate the similarity between the model's output data and historical sample data. The loss function is a function of the difference between the model's output data and historical sample data, and the model's weight parameters can be adjusted by taking the derivative of the loss function.

[0136] Here, the loss function can be the mean squared error function, the binary cross-entropy function, or the cross-entropy function, etc. This disclosure does not limit the specific choice of the loss function.

[0137] Taking mean squared error as the loss function as an example, the summation and averaging of the squared differences between the model output data and the historical sample data can be calculated. The smaller the mean squared error, the smaller the error between the model output data and the historical sample data, and the closer the output touch coordinates are to the historical touch coordinates in the historical sample data.

[0138] In this embodiment of the disclosure, the determination of touch coordinates can be summarized as follows: Figure 4 The process steps are as follows: Figure 4 This is a schematic diagram illustrating the determination of touch coordinates according to an exemplary embodiment, such as... Figure 4As shown, the determination of touch coordinates in this embodiment can be divided into three steps: the first step is model establishment, the second step is model training, and the third step is model usage.

[0139] In the first step of model building, historical sensing data can be collected based on multiple inertial sensors during the historical use of the mobile terminal, and historical touch coordinates corresponding to the historical sensing data can be obtained. The historical sensing data and the historical touch coordinates corresponding to the historical sensing data are combined to form historical sample data.

[0140] In the second step of model building, the selected initial prediction model is trained based on the historical sample data from the first step; for example, the selected initial prediction model can be a BP model. Before training, the historical sample data is preprocessed to remove erroneous data. Then, the historical sensor data and corresponding labels (historical touch coordinates) from the preprocessed historical sample data are input into the BP model, and the touch coordinates are output. During training, the model parameters are continuously optimized based on the output results to obtain the target prediction model.

[0141] In the third step of model usage, sensor data is read in real time, and the sensor data is input into the target prediction model to output touch coordinates.

[0142] In this way, by iteratively processing the initial prediction model to be trained using historical sample data collected during historical use, a target prediction model that can accurately predict touch coordinates can be obtained, providing a foundation for the accurate determination of subsequent touch coordinates and corresponding functional responses.

[0143] This disclosure also provides a touch device. Figure 5 This is a schematic diagram illustrating the structure of a touch device according to an exemplary embodiment, such as... Figure 5 As shown, the touch device 500 includes:

[0144] The data acquisition module 501 is used to acquire sensing data collected by at least two inertial sensors in the mobile terminal, wherein the at least two inertial sensors are used to generate the sensing data based on touch actions on the mobile terminal.

[0145] The coordinate generation module 502 is used to generate the touch coordinates of the touch action based on the sensing data;

[0146] The response module 503 is used to respond to the touch action based on the touch coordinates.

[0147] In some embodiments, the response module 503 is further configured to respond to the touch action when the touch coordinates meet a preset position triggering condition.

[0148] In some embodiments, the response module 503 is further configured to respond to the touch action when the current area of ​​the touch coordinates matches a preset active area.

[0149] In some embodiments, the apparatus further includes:

[0150] The region acquisition module is used to acquire the preset sub-operating regions corresponding to various types of touch actions; different types of touch actions correspond to different preset sub-operating regions.

[0151] The response module 503 is further configured to:

[0152] When the area where the touch coordinates are currently located matches the preset sub-area corresponding to the touch action, the touch action is responded to.

[0153] In some embodiments, the data acquisition module is further configured to acquire sensing data collected by at least two inertial sensors within the mobile terminal, which are collected by touch actions acting on the back cover of the mobile terminal.

[0154] In some embodiments, the coordinate generation module is further configured to input the sensing data into a preset target prediction model to obtain the touch coordinates of the touch action.

[0155] In some embodiments, the apparatus further includes:

[0156] The sample acquisition module is used to acquire historical sample data collected during the historical use of the mobile terminal, wherein the historical sample data includes: historical sensor data and historical touch coordinates corresponding to the historical sensor data;

[0157] The iterative processing module is used to input the historical sample data into the initial prediction model to be trained for iterative processing until the difference between the output touch coordinates and the historical touch coordinates in the historical sample data satisfies the convergence condition, thereby obtaining the target prediction model.

[0158] In some embodiments, the data acquisition module is further configured to acquire sensing data based on a first inertial sensor located in a first region and a second inertial sensor located in a second region on the back cover of the mobile terminal; wherein the first region and the second region are distributed on both sides of an axis parallel to the short side direction of the mobile terminal.

[0159] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0160] Figure 6This is a block diagram illustrating a touch device 1800 according to an exemplary embodiment. For example, device 1800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness device, personal digital assistant, etc.

[0161] Reference Figure 6 The device 1800 may include one or more of the following components: a processing component 1802, a memory 1804, a power component 1806, a multimedia component 1808, an audio component 1810, an input / output (I / O) interface 1812, a sensor component 1814, and a communication component 1816.

[0162] Processing component 1802 typically controls the overall operation of device 1800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 1802 may include one or more processors 1820 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 1802 may also include one or more modules to facilitate interaction between processing component 1802 and other components. For example, processing component 1802 may include a multimedia module to facilitate interaction between multimedia component 1808 and processing component 1802.

[0163] Memory 1804 is configured to store various types of data to support the operation of device 1800. Examples of this data include instructions for any application or method operating on device 1800, contact data, phonebook data, messages, images, videos, etc. Memory 1804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0164] The power supply component 1806 provides power to the various components of the device 1800. The power supply component 1806 may include: a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 1800.

[0165] Multimedia component 1808 includes a screen that provides an output interface between the device 1800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 1808 includes a front-facing camera and / or a rear-facing camera. When the device 1800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and / or rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0166] Audio component 1810 is configured to output and / or input audio signals. For example, audio component 1810 includes a microphone (MIC) configured to receive external audio signals when device 1800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 1804 or transmitted via communication component 1816. In some embodiments, audio component 1810 also includes a speaker for outputting audio signals.

[0167] I / O interface 1812 provides an interface between processing component 1802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0168] Sensor assembly 1814 includes one or more sensors for providing status assessments of various aspects of device 1800. For example, sensor assembly 1814 may detect the on / off state of device 1800, the relative positioning of components such as the display and keypad of device 1800, changes in the position of device 1800 or a component of device 1800, the presence or absence of user contact with device 1800, the orientation or acceleration / deceleration of device 1800, and temperature changes of device 1800. Sensor assembly 1814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 1814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 1814 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0169] Communication component 1816 is configured to facilitate wired or wireless communication between device 1800 and other devices. Device 1800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 1816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 1816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, or other technologies.

[0170] In an exemplary embodiment, the apparatus 1800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0171] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 1804 including instructions, which can be executed by a processor 1820 of the device 1800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0172] A non-transitory computer-readable storage medium, wherein instructions in the storage medium, when executed by a processor, enable the execution of the above-described method.

[0173] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

[0174] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A touch method, characterized in that, Applied to a mobile terminal, comprising: obtaining sensing data collected by at least two inertial sensors in the mobile terminal, wherein the at least two inertial sensors are configured to generate the sensing data according to a touch action on the mobile terminal; generating touch coordinates of the touch action based on the sensing data; responding to the touch action based on the touch coordinates; the obtaining sensing data collected by at least two inertial sensors in the mobile terminal comprises: in response to executing an application on the display screen of the mobile terminal, obtaining sensing data generated by a touch action on the back cover of the mobile terminal.

2. The method of claim 1, wherein, the responding to the touch action based on the touch coordinates comprises: responding to the touch action when the touch coordinates meet a preset position trigger condition.

3. The method of claim 2, wherein, the responding to the touch action when the touch coordinates meet a preset position trigger condition comprises: responding to the touch action when the current area of the touch coordinates matches a preset action area.

4. The method of claim 3, wherein, the method further comprises: obtaining preset sub-action areas corresponding to each type of touch action, wherein different types of touch actions correspond to different preset sub-action areas; the responding to the touch action when the current area of the touch coordinates matches a preset action area comprises: responding to the touch action when the current area of the touch coordinates matches a preset sub-action area corresponding to the touch action.

5. The method of claim 1, wherein, the generating touch coordinates of the touch action based on the sensing data comprises: inputting the sensing data into a preset target prediction model to obtain the touch coordinates of the touch action.

6. The method of claim 5, wherein, the method further comprises: obtaining historical sample data collected by the mobile terminal, wherein the historical sample data includes historical sensing data and historical touch coordinates corresponding to the historical sensing data; inputting the historical sample data into an initial prediction model to be trained for iterative processing until the difference between the output touch coordinates and the historical touch coordinates in the historical sample data meets a convergence condition, obtaining the target prediction model.

7. The method according to any one of claims 1 to 6, characterized in that, the obtaining sensing data collected by at least two inertial sensors in the mobile terminal comprises: obtaining sensing data from a first inertial sensor located in a first area and a second inertial sensor located in a second area on the back cover of the mobile terminal, wherein the first area and the second area are distributed on both sides of an axis of the mobile terminal parallel to the short side direction.

8. A touch device, comprising: Applied to a mobile terminal, comprising: a data acquisition module configured to obtain sensing data collected by at least two inertial sensors in the mobile terminal, wherein the at least two inertial sensors are configured to generate the sensing data according to a touch action on the mobile terminal; a coordinate generation module configured to generate touch coordinates of the touch action based on the sensing data; a response module configured to respond to the touch action based on the touch coordinates; The data acquisition module is further configured to acquire, in response to execution of an application program on a display screen of the mobile terminal, sensing data generated by a touch action on the back cover of the mobile terminal and collected by the at least two inertial sensors.

9. The apparatus of claim 8, wherein, The response module is further configured to respond to the touch action when the touch coordinates satisfy a preset position triggering condition.

10. The apparatus of claim 9, wherein, The response module is further configured to respond to the touch action when the current region of the touch coordinates matches the preset action region.

11. The apparatus of claim 10, wherein, The apparatus further includes: a region acquisition module configured to acquire preset sub-action regions corresponding to various types of touch actions, wherein different types of touch actions correspond to different preset sub-action regions; The response module is further configured to: respond to the touch action when the current region of the touch coordinates matches the preset sub-action region corresponding to the touch action.

12. The device of any one of claims 8-11, wherein, The data acquisition module is further configured to acquire sensing data from a first inertial sensor located in a first region and a second inertial sensor located in a second region on the back cover of the mobile terminal, wherein the first region and the second region are distributed on both sides of an axis of the mobile terminal parallel to the short side direction.

13. A touch device, comprising: The apparatus further includes: a processor and a memory configured to store executable instructions capable of running on the processor, wherein: when the processor runs the executable instructions, the executable instructions perform the steps in the method provided in any one of claims 1 to 7.

14. A non-transitory computer-readable storage medium, comprising: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions, when executed by a processor, implement the steps in the method provided in any one of claims 1 to 7.

Images