An operation detection method, readable storage medium and electronic device

Abstract

This application provides an operation detection method, a readable storage medium, and an electronic device, belonging to the field of touch technology. The method is applied to an electronic device including multiple accelerometers. In this method, the electronic device can collect acceleration data corresponding to a user's click operation through the multiple accelerometers, and determine whether each accelerometer is abnormal based on the acceleration data collected by each accelerometer. Then, the electronic device can use the acceleration data collected by one or more of the accelerometers that are not abnormal to identify whether the user's click operation is a knuckle operation. This avoids the impact of accelerometer malfunctions on the accuracy of the electronic device's knuckle operation recognition.

Description

Technical Field

[0001] This application relates to the field of touch technology, and more particularly to an operation detection method, a readable storage medium, and an electronic device. Background Technology

[0002] Knuckle tapping refers to actions performed by a user on a touchscreen using their finger knuckles, such as knuckle taps. When a user taps the touchscreen, the electronic device uses an accelerometer to collect acceleration data and determines whether the tap was a knuckle tap or a full finger tap based on this data.

[0003] However, in cases where the electronic device is large and only has one accelerometer, the acceleration data collected by that single accelerometer may not reflect the acceleration data when the user taps the touchscreen, resulting in a decrease in the accuracy of the electronic device in recognizing knuckle gestures. Summary of the Invention

[0004] This application provides an operation detection method, a readable storage medium, and an electronic device, which helps improve the accuracy of electronic devices in recognizing knuckle operations.

[0005] In a first aspect, an operation detection method is provided, applied to an electronic device, the electronic device including a first accelerometer and a second accelerometer; and the method includes: detecting a first operation by a user; if the first accelerometer is normal and the second accelerometer is abnormal, identifying whether the first operation is a knuckle operation using first acceleration data collected by the first accelerometer; and if the first accelerometer is abnormal and the second accelerometer is normal, identifying whether the first operation is a knuckle operation using second acceleration data collected by the second accelerometer.

[0006] In this method, the electronic device can identify whether the first operation is a knuckle operation based on acceleration data collected by the accelerometer sensor (which is free of anomalies) between the first and second accelerometer sensors. This avoids using acceleration data collected by an accelerometer sensor exhibiting anomalies to identify whether the first operation is a knuckle operation, thus improving the accuracy of the electronic device in recognizing knuckle operations.

[0007] In one possible implementation of the first aspect above, the aforementioned anomaly includes: in one or more directions of acceleration data collected by the accelerometer, the absolute value of acceleration data in at least one direction is greater than a first threshold for a continuous first time period; or, in one or more directions of acceleration data collected by the accelerometer, the absolute value of acceleration data in at least one direction is greater than a second threshold for a continuous first number of acceleration data.

[0008] In this implementation, the acceleration data collected by the accelerometer includes acceleration data in at least one direction (at least one axis). If, during the acceleration data collection process, an accelerometer collects acceleration data in a series of acceleration data collections within a first consecutive time period, the absolute value of acceleration data in at least one direction is greater than a first threshold, the accelerometer can be determined to be malfunctioning. Similarly, if, during the acceleration data collection process, an accelerometer collects acceleration data in a series of acceleration data collections within a first consecutive number of times, the absolute value of acceleration data in at least one direction is greater than the first threshold, the accelerometer can be determined to be malfunctioning.

[0009] In one possible implementation of the first aspect above, the method further includes: if the first operation is identified as a knuckle operation, executing the business corresponding to the knuckle operation.

[0010] In one possible implementation of the first aspect above, the above further includes: when the first accelerometer and the second accelerometer are normal, identifying whether the first operation is a knuckle operation based on the acceleration data collected by the sensor closest to the operation position of the first operation among the first accelerometer and the second accelerometer; or, when both the first accelerometer and the second accelerometer are abnormal, identifying whether the first operation is a knuckle operation based on the acceleration data collected by the sensor closest to the operation position of the first operation among the first accelerometer and the second accelerometer.

[0011] In this implementation, if both the first and second accelerometers are functioning normally or abnormally, the electronic device identifies whether the first operation is a knuckle operation based on the acceleration data collected by the sensor closest to the operation position of the first operation. Since the closer the accelerometer is to the operation position, the more accurately the collected acceleration data reflects the acceleration corresponding to the first operation, the accuracy of the electronic device in recognizing knuckle operations can be improved.

[0012] In one possible implementation of the first aspect described above, the electronic device is a foldable electronic device, with a first accelerometer disposed on one side of the folding axis of the foldable electronic device and a second accelerometer disposed on the other side of the folding axis; and the method further includes: if the first operation is an operation in the screen area on one side and the first and second accelerometers are functioning normally, identifying whether the first operation is a knuckle operation based on the first acceleration data; or, if the first operation is an operation in the screen area on the other side and the first and second accelerometers are functioning normally, identifying whether the first operation is a knuckle operation based on the second acceleration data.

[0013] In one possible implementation of the first aspect described above, the method of identifying whether a first operation is a knuckle operation using the first acceleration data collected by the first acceleration sensor when the first acceleration sensor is normal and the second acceleration sensor is abnormal includes: acquiring a first status indicator of the first acceleration sensor and a second status indicator of the second acceleration sensor; and identifying whether a first operation is a knuckle operation using the first acceleration data when the first status indicator indicates that the first acceleration sensor is normal and the second status indicator indicates that the second acceleration sensor is abnormal.

[0014] In this implementation, different values ​​of the status flag can indicate whether the accelerometer is malfunctioning.

[0015] In one possible implementation of the first aspect described above, the electronic device includes an accelerometer driver, and the first and second state identifiers are maintained by the accelerometer driver.

[0016] In this implementation, the first state identifier of the first accelerometer and the second state identifier of the second accelerometer are determined by the accelerometer driver, so that other modules can determine whether the first accelerometer and the second accelerometer are abnormal by using the first state identifier and the second state identifier, respectively.

[0017] In one possible implementation of the first aspect described above, the accelerometer driver maintains the second status flag in the following manner: when the accelerometer driver detects an anomaly in the first accelerometer, it configures the first status flag to indicate that the first accelerometer is abnormal; when the accelerometer driver detects an anomaly in the second accelerometer, it configures the second status flag to indicate that the second accelerometer is abnormal; when the accelerometer driver detects that the first accelerometer is not abnormal, it configures the first status flag to indicate that the first accelerometer is normal; when the accelerometer driver detects that the second accelerometer is not abnormal, it configures the second status flag to indicate that the second accelerometer is normal.

[0018] In one possible implementation of the first aspect above, the electronic device includes a knuckle detection driver and a knuckle detection hardware abstraction layer; and, when a first state indicator indicates that the first accelerometer is normal and a second state indicator indicates that the second accelerometer is abnormal, identifying whether the first operation is a knuckle operation using the first acceleration data includes: the knuckle detection driver sending the first acceleration data to the knuckle detection hardware abstraction layer when the first state indicator indicates that the first accelerometer is normal and the second state indicator indicates that the second accelerometer is abnormal; and the knuckle detection hardware abstraction layer using the first acceleration data to identify whether the first operation is a knuckle operation.

[0019] In this implementation, the knuckle detection driver can determine whether the first and second accelerometers are malfunctioning based on a first and a second state identifier, respectively, and report the acceleration data collected by the accelerometer that is not malfunctioning to the knuckle detection hardware abstraction layer. This allows the knuckle detection hardware abstraction layer to identify whether the first operation is a knuckle operation based on the acceleration data collected by the accelerometer that is not malfunctioning.

[0020] In one possible implementation of the first aspect above, the electronic device further includes at least one third acceleration sensor; and the method further includes: in the event of an anomaly in the first acceleration sensor and the second acceleration sensor, using third acceleration data collected by at least one third acceleration sensor to identify whether the first operation is a knuckle operation.

[0021] In this implementation, if the first and second accelerometers malfunction, the electronic device can identify whether the first operation is a knuckle operation based on the third acceleration data collected by the third accelerometer, which is not malfunctioning.

[0022] In a second aspect, an operation detection method is provided, applied to an electronic device, the electronic device including a plurality of acceleration sensors; and the method includes: acquiring acceleration data collected by each acceleration sensor; and performing a first service based on the acceleration data collected by one or more acceleration sensors among the plurality of acceleration sensors, wherein no abnormality has occurred.

[0023] In this method, the electronic device can perform the first service based on acceleration data collected by an accelerometer among multiple accelerometers that is free of anomalies. This avoids using acceleration data collected by an accelerometer that is malfunctioning, thus improving the accuracy of the electronic device in performing the first service.

[0024] In one possible implementation of the second aspect above, the aforementioned anomaly includes: in the acceleration data collected by the accelerometer, the absolute value of acceleration data in at least one direction is greater than a first threshold for a continuous first time period; or, in the acceleration data collected by the accelerometer, the absolute value of a continuous first number of acceleration data in at least one direction is greater than a second threshold.

[0025] In one possible implementation of the second aspect above, the first service includes at least one of the following services: a service for detecting whether a user's first operation on an electronic device is a knuckle operation, a service for adjusting the display orientation of the electronic device, and a service for recording motion information of the electronic device.

[0026] Thirdly, an electronic device is provided, comprising: one or more processors; one or more memories; a plurality of acceleration sensors; wherein the one or more memories store one or more programs, which, when executed by the one or more processors, cause the electronic device to perform any of the methods provided in the first or second aspect above.

[0027] Fourthly, a computer-readable storage medium is provided, which stores one or more programs that, when executed on an electronic device, cause the electronic device to perform any of the methods provided in the first or second aspect above.

[0028] Fifthly, a computer program product is provided that, when run on an electronic device, causes the electronic device to perform any one of the methods provided in the first or second aspect above.

[0029] It should be understood that the beneficial effects of the second to fifth aspects mentioned above can be referred to the description of the first aspect, and will not be repeated here. Attached Figure Description

[0030] Figure 1A According to some embodiments of this application, a schematic diagram of acceleration data collected by a mobile phone 10 in different situations for recognizing knuckle operations is shown.

[0031] Figure 1B According to some embodiments of this application, a schematic diagram of a mobile phone 10 taking a screenshot in response to a knuckle gesture is shown.

[0032] Figure 2 According to some embodiments of this application, a schematic diagram of a mobile phone 10 recognizing knuckle operations based on an IMU without any abnormalities is shown.

[0033] Figure 3 According to some embodiments of this application, a schematic diagram of an operation detection method is shown.

[0034] Figure 4 According to some embodiments of this application, a schematic diagram of the system architecture of a mobile phone 10 is shown.

[0035] Figure 5A According to some embodiments of this application, a schematic flowchart of an acceleration sensor drive maintaining the state of an acceleration sensor is shown.

[0036] Figure 5B According to some embodiments of this application, a schematic diagram of another process for maintaining the state of an accelerometer sensor is shown.

[0037] Figure 5CAccording to some embodiments of this application, a schematic flowchart of another acceleration sensor drive maintaining the state of the acceleration sensor is shown.

[0038] Figure 6 According to some embodiments of this application, a schematic diagram of a process is shown in which a knuckle detection driver reports acceleration data to the knuckle detection HAL based on the state of an accelerometer sensor.

[0039] Figure 7 According to some embodiments of this application, an interactive flowchart of an operation detection method is shown.

[0040] Figure 8 According to some embodiments of this application, a schematic flowchart of another operation detection method is shown.

[0041] Figure 9 According to some embodiments of this application, a flowchart of yet another operation detection method is shown.

[0042] Figure 10 According to some embodiments of this application, a structural schematic diagram of a mobile phone 10 is shown. Detailed Implementation

[0043] The embodiments of this application include, but are not limited to, operation detection methods, readable storage media, and electronic devices.

[0044] It should be noted that the technical solutions provided in this application embodiment can be applied to any electronic device including multiple acceleration sensors, including but not limited to mobile stations (MS), mobile terminals (MT), etc. The electronic device can be a mobile phone, smart TV, wearable device, tablet computer, computer with wireless transceiver function, virtual reality (VR) terminal device, augmented reality (AR) terminal device, wireless terminal in industrial control, wireless terminal in self-driving, wireless terminal in remote medical surgery, wireless terminal in smart grid, wireless terminal in transportation safety, wireless terminal in smart city, wireless terminal in smart home, etc. This application embodiment does not limit the specific form of the electronic device. For ease of description, the following uses a foldable mobile phone (hereinafter referred to as mobile phone 10) as an example to introduce the technical solution of this application.

[0045] The technical solutions of the embodiments of this application are described below with reference to the accompanying drawings.

[0046] As mentioned earlier, when an electronic device is large in size and only has one accelerometer, the acceleration data collected by that single accelerometer may not reflect the acceleration data when the user taps the touchscreen, resulting in a decrease in the accuracy of the electronic device in recognizing knuckle operations.

[0047] To improve the accuracy of electronic devices in recognizing knuckle gestures, in some embodiments, multiple accelerometers can be installed in the electronic device, and the user's tap gesture can be identified as a knuckle gesture or a finger gesture based on the acceleration data collected by the accelerometer closest to the tap position.

[0048] For example, Figure 1A According to some embodiments of this application, a schematic diagram of acceleration data collected by a mobile phone 10 in different situations for recognizing knuckle operations is shown.

[0049] like Figure 1A As shown, the mobile phone 10 includes a flexible screen 11 and a non-flexible screen 12. The flexible screen 11 can be folded along the folding axis NN (the axis NN is parallel to the Y direction). An inertial measurement unit (IMU) (an accelerometer) 01 is disposed on the left side of the folding axis NN (on one side of the display area 11A of the flexible screen 11), and an IMU 02 is disposed on the right side of the folding axis NN (on one side of the display area 11B of the flexible screen 11).

[0050] When a user clicks on display area 11A or on non-flexible screen 12, the mobile phone 10 can identify whether the user's click is a knuckle operation based on the acceleration data collected by IMU 01; when a user clicks on display area 11B, the mobile phone 10 can identify whether the user's click is a knuckle operation based on the acceleration data collected by IMU 02.

[0051] In some embodiments, after recognizing that the user's tap operation is a knuckle tap, the mobile phone 10 can execute the service corresponding to the knuckle tap operation. For example, refer to Figure 1B When the phone recognizes that the user's tap is a double tap with a knuckle, it can take a screenshot of the screen displayed on the phone.

[0052] based on Figure 1AAs shown, the mobile phone 10 can use acceleration data collected by the accelerometer that is closest to the click position of the user's click operation to identify whether the user's click operation is a knuckle operation, which can improve the accuracy of the knuckle operation recognized by the mobile phone 10.

[0053] However, accelerometers are microelectromechanical systems (MEMS) devices with intricate and complex internal micromechanical structures. Under excessive external stress or structural failure, these micromechanical structures can adhere, causing the acceleration data collected by the accelerometer to fluctuate around a certain value (usually much higher than normal) (a condition known as "axis jamming"). Thus, if the accelerometer with the smallest distance to the user's click position is jammed, using its acceleration data to identify whether the user's click is a knuckle gesture will lead to incorrect results (e.g., misidentifying a finger click as a knuckle gesture, or vice versa).

[0054] Based on this, embodiments of this application provide an operation detection method applied to an electronic device including multiple accelerometers. In this method, the electronic device can collect acceleration data corresponding to a user's click operation using the multiple accelerometers, and determine whether each accelerometer is malfunctioning (e.g., whether it is stuck). Then, the electronic device can use the acceleration data collected by one or more accelerometers without malfunctions (e.g., the accelerometer with the smallest distance to the click position of the user's click operation among the malfunction-free accelerometers) to identify whether the user's click operation is a knuckle operation.

[0055] This avoids the accelerometer malfunction affecting the accuracy of electronic devices in recognizing knuckle movements.

[0056] For example, regarding the aforementioned mobile phone 10, refer to Figure 2 :

[0057] When the user's click operation is performed in display area 11A (or on non-flexible screen 12), and IMU 01 and IMU 02 are functioning normally (or abnormally), the mobile phone 10 can identify whether the user's click operation is a knuckle operation based on the acceleration data collected by IMU 01; when the user's click operation is performed in display area 11B, and IMU 01 and IMU 02 are functioning normally, the mobile phone 10 can identify whether the user's click operation is a knuckle operation based on the acceleration data collected by IMU 02.

[0058] Continue to refer to Figure 2 When the user's click operation is performed in display area 11A (or on the non-flexible screen 12), IMU 01 is malfunctioning, and IMU 02 is functioning normally, the mobile phone 10 can identify whether the user's click operation is a knuckle operation based on the acceleration data collected by IMU 02. When the user's click operation is performed in display area 1B, IMU 01 is functioning normally, and IMU 02 is malfunctioning, the mobile phone 10 can identify whether the user's click operation is a knuckle operation based on the acceleration data collected by IMU 01.

[0059] Therefore, even if either IMU 01 or IMU 02 malfunctions, the phone 10 can still identify whether the user's tap operation is a knuckle tap based on the acceleration data collected by the other IMU. This avoids the phone 10 using acceleration data collected by the malfunctioning IMU to identify whether the user's tap operation is a knuckle tap, thus improving the accuracy of the phone 10 in recognizing knuckle taps.

[0060] It should be noted that an acceleration sensor can be any device capable of detecting acceleration, including but not limited to IMU, accelerometer, gyroscope, etc.

[0061] In some embodiments, the mobile phone 10 can determine whether an accelerometer is malfunctioning based on the relationship between the acceleration data collected by each accelerometer and an acceleration threshold. For example, the acceleration data collected by the accelerometer may include acceleration data from multiple axes (i.e., multiple directions corresponding one-to-one with the axes). The mobile phone 10 can determine that the accelerometer is malfunctioning if, in the acceleration data collected by an accelerometer, the absolute value of the acceleration data for any axis is greater than a first acceleration threshold within a first time period, or if the absolute values ​​of the acceleration data for any axis for a consecutive first number of times are greater than a second acceleration threshold. The first acceleration threshold and the second acceleration threshold may be the same or different.

[0062] It should be noted that the first duration and the first quantity can be empirical values ​​or experimental values. For example, the first duration can be 0.5 seconds, 0.8 seconds, 1 second, 2 seconds, or any other arbitrary duration.

[0063] It should be noted that the first and second acceleration thresholds can be empirical or experimental values. For example, the first and second acceleration thresholds can be any values ​​greater than gravitational acceleration, such as K times the gravitational acceleration, where K is a real number greater than 1. For example, K can be 3, 4, 5, 6, 7, or other real numbers.

[0064] In some embodiments, each accelerometer in the electronic device may correspond to a display area. After detecting a user's click operation in a certain display area, if the accelerometer corresponding to that display area is functioning normally, the electronic device uses the acceleration data collected by that accelerometer to identify whether the click operation is a knuckle operation.

[0065] It should be noted that knuckle operation can be any operation performed on the display screen of an electronic device using the knuckle, including but not limited to knuckle tapping, knuckle double tapping, knuckle multiple taps, and knuckle swiping.

[0066] It should be noted that the operating system or applications of electronic devices have pre-set functions for different knuckle gestures. After recognizing a knuckle gesture, the operating system or applications of the electronic device can execute the relevant functions. For example, after recognizing a double-knuckle gesture, the electronic device can take a screenshot of the interface displayed on the electronic device.

[0067] In some embodiments, the driver program for the accelerometers in the electronic device (hereinafter referred to as the accelerometer driver) can detect the state (normal or abnormal) of each accelerometer and configure an interface (e.g., an application programming interface, API) for acquiring the state of each accelerometer. Other applications, drivers, services, the operating system, or other modules in the electronic device can obtain the state of each accelerometer through the API and perform related services based on the data collected by the accelerometers in normal state. Based on this, embodiments of this application also provide an operation detection method, in which the electronic device can collect acceleration data through the multiple accelerometers respectively, and determine whether each accelerometer is abnormal (e.g., whether there is a jammed axis) based on the acceleration data collected by each accelerometer. Then, the electronic device can use the acceleration data collected by one or more accelerometers among the accelerometers without abnormalities to perform related services, such as identifying whether the user's click operation is a knuckle operation, adjusting the display direction of the electronic device, recording the motion information of the electronic device, etc.

[0068] Below, in conjunction with Figures 3 to 10 The technical solution of this application is described.

[0069] For example, Figure 3 According to some embodiments of this application, a schematic diagram of an operation detection method is shown. The entity performing the operation is a mobile phone 10, such as... Figure 3 As shown, the method includes the following steps:

[0070] S301, the user's first action was detected.

[0071] For example, the mobile phone 10 can trigger the operation detection method of this application embodiment when it detects a first operation by the user on the display screen of the mobile phone 10.

[0072] For example, the first operation could be a user tapping, swiping, or pinching operation on any display area (or position) of any screen on the phone 10 using their finger, knuckle, or other body parts or objects (such as a stylus).

[0073] In other embodiments, the first operation may also be any other operation performed on any display area (or position) of any display screen of the mobile phone 10, and is not limited here.

[0074] S302, acquire the first acceleration data corresponding to the first operation collected by IMU 01 and the second acceleration data corresponding to the first operation collected by IMU 02.

[0075] After detecting the user's first operation, the mobile phone 10 can acquire the first acceleration data corresponding to the first operation collected by IMU 01 and the second acceleration data corresponding to the first operation collected by IMU 02.

[0076] In some embodiments, IMU 01 and IMU 02 can continuously collect acceleration data or collect acceleration data when the user's first operation is detected. Mobile phone 10 can acquire the first acceleration data collected by IMU 01 and the second acceleration data collected by IMU 02 within the time (or time period) of the first operation.

[0077] After acquiring the first acceleration data and the second acceleration data corresponding to the first operation, the mobile phone 10 can use the acceleration data collected by the IMU 01 and IMU 02, which are free of abnormalities, to identify whether the first operation is a knuckle operation.

[0078] In some embodiments, the mobile phone 10 can obtain the status of IMU 01 and IMU 02 from the accelerometer sensor driver, which will be described in detail below and will not be repeated here.

[0079] It should be noted that the method by which the phone determines the status (normal or abnormal) of IMU 01 or IMU 02 will be described in detail below, and will not be repeated here.

[0080] S303A, when IMU 01 is abnormal and IMU 02 is normal, identifies whether the first operation is a knuckle operation based on the second acceleration data.

[0081] For example, if IMU 01 is abnormal and IMU 02 is normal, then the first acceleration data is abnormal data and the second acceleration data is normal data. The mobile phone 10 can identify whether the first operation is a knuckle operation based on the second acceleration data.

[0082] S303B, when IMU 01 is normal and IMU 02 is abnormal, identifies whether the first operation is a knuckle operation based on the first acceleration data.

[0083] For example, if IMU 01 is normal and IMU 02 is abnormal, then the first acceleration data is normal data and the second acceleration data is abnormal data. The mobile phone 10 can identify whether the first operation is a knuckle operation based on the first acceleration data.

[0084] S303C, when both IMU 01 and IMU 02 are abnormal or both IMU 01 and IMU 02 are normal, identifies whether the first operation is a knuckle operation based on the acceleration data collected by the IMU with the smallest operating position of the first operation among IMU 01 and IMU 02.

[0085] For example, if both IMU 01 and IMU 02 are abnormal, then both the first acceleration data and the second acceleration data are abnormal. The mobile phone 10 can identify whether the first operation is a knuckle operation based on the acceleration data collected by the IMU with the closest operating position to the first operation. Thus, even though both the first and second acceleration data are abnormal, the closer the data is to the operating position of the first operation, the more accurately the IMU's data reflects the acceleration of the first operation, thereby improving the accuracy of knuckle operation recognition.

[0086] For example, if the first operation occurs in display area 11A and both IMU 01 and IMU 02 are malfunctioning, the mobile phone 10 can identify whether the first operation is a knuckle operation based on the first acceleration data. If the first operation occurs in display area 11B or on the non-flexible screen 12 and both IMU 01 and IMU 02 are malfunctioning, the mobile phone 10 can identify whether the first operation is a knuckle operation based on the second acceleration data.

[0087] For example, if both IMU 01 and IMU 02 are functioning normally, then both the first acceleration data and the second acceleration data are normal. The mobile phone 10 can identify whether the first operation is a knuckle operation based on the acceleration data collected by the IMU with the smallest distance from the operation position of the first operation. Thus, since the smaller the distance between the IMU and the operation position of the first operation, the more accurately the data collected by the IMU reflects the acceleration of the first operation, the higher the accuracy of knuckle operation recognition can be.

[0088] It should be noted that if the phone 10 has other accelerometers, even if both IMU 01 and IMU 02 are malfunctioning, the phone 10 can still identify whether the first operation is a knuckle operation based on the acceleration data collected by other accelerometers that are not malfunctioning.

[0089] It should be noted that if the first operation is identified as a knuckle gesture, the phone can execute the corresponding service, such as taking a screenshot or recording the screen.

[0090] Based on the above method, when IMU 01 is malfunctioning, mobile phone 10 can use the acceleration data collected by IMU 02 to identify whether the first operation is a knuckle operation, and when IMU 02 is malfunctioning, it can use the acceleration data collected by IMU 01 to identify whether the first operation is a knuckle operation. In this way, the accuracy of mobile phone 10 in recognizing knuckle operations can be improved.

[0091] The technical solution of this application will be introduced below, taking into account the system architecture of the mobile phone 10.

[0092] For example, Figure 4 According to some embodiments of this application, a schematic diagram of a system architecture for a mobile phone 10 is shown.

[0093] like Figure 4 As shown, the software of mobile phone 10 includes an application layer, a framework layer, a hardware abstraction layer, and a universal smart sensor hub (Sensorhub), while the hardware includes IMU 01 and IMU 02.

[0094] The application layer includes applications for the phone 10, such as applications that support knuckle gestures (referred to as knuckle apps), such as screenshot apps, etc.

[0095] The framework layer (also known as the application framework layer) provides the application programming interface (API) and programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0096] For example, the framework layer may include a finger sense service, which provides APIs for finger sense applications to register, invoke, and return finger sense recognition results for finger sense operation detection. Finger sense applications can register with the finger sense service to listen to the finger sense recognition results obtained from the hardware abstract layer (HAL) of finger sense detection.

[0097] In other embodiments, the framework layer also includes an input manager service (IMS) for triggering a knuckle detection service, a knuckle detection HAL, a knuckle detection driver, etc., to identify whether the user's operation is a knuckle operation when a user's operation on the electronic device is detected.

[0098] The Hardware Abstraction Layer (HAL) is the interface layer between the operating system kernel and the hardware of the mobile phone 10, and can include hardware abstraction interfaces. For example, the HAL can include the knuckle detection HAL (fingersense-HAL).

[0099] The knuckle detection HAL is used to report acceleration data to the knuckle detection driver based on the call instructions of the knuckle detection service, identify whether the user operation is a knuckle operation based on the acceleration data reported by the knuckle detection driver, and pass the knuckle detection results to the knuckle application through the knuckle detection service.

[0100] In some embodiments, the acceleration data reported by the knuckle detection driver to the knuckle detection HAL may include two sets of acceleration data. The knuckle detection HAL can identify whether a user operation is a knuckle operation based on one or more of these two sets of acceleration data.

[0101] For example, in the event of an IMU 01 malfunction, the two acceleration data sets are identical, both being acceleration data from IMU 02. The knuckle detection HAL can identify whether a user's operation is a knuckle gesture based on any one of the acceleration data sets.

[0102] For example, in the case of an IMU 02 malfunction, the two acceleration data sets are identical, both being acceleration data from IMU 01. The knuckle detection HAL can identify whether a user's operation is a knuckle gesture based on any one of the acceleration data sets.

[0103] For example, if both IMU 01 and IMU 02 are functioning normally, the two acceleration data sets are the acceleration data of IMU 01 and IMU 02, respectively. If the user's operation is located in display area 11A or the non-flexible screen 12, the knuckle detection HAL can identify whether the user's operation is a knuckle operation based on the acceleration data of IMU 01; if the user's operation is located in display area 11B, the knuckle detection HAL can identify whether the user's operation is a knuckle operation based on the acceleration data of IMU 02.

[0104] Sensorhub includes drivers for various sensors or functional modules in the phone, such as knuckle detection drivers and accelerometer drivers.

[0105] The accelerometer driver is used to acquire acceleration data collected by IMU 01 (IMU01 data) and acceleration data collected by IMU 02 (IMU 02 data) from IMU 01 and IMU 02 respectively, and transmit the IMU 01 data and IMU 02 data to other modules (such as the knuckle detection driver).

[0106] The accelerometer driver is also used to determine whether there are any abnormalities in IMU01 and IMU02 based on the acceleration data collected by IMU01 and IMU02, and to configure the status flags of IMU01 and IMU02 (to indicate whether there are any abnormalities in IMU01 and IMU02) for use by other modules.

[0107] For example, the accelerometer driver can maintain a state management array that records the sensor identifier and state identifier of the accelerometer in the mobile phone 10 that indicates an accelerometer malfunction. The state identifier is initially a normal value, and it is a third value when the state identifier indicates that an accelerometer is malfunctioning. In some embodiments, if the sensor identifier of an accelerometer is in the state management data group, then the accelerometer is stored as malfunctioning.

[0108] For example, the status management array can be a multi-row, two-column array. One row stores the sensor identifier (e.g., stored in the first column of the row) and the status identifier (e.g., stored in the second column of the row, with the value being the third value) of an accelerometer that is malfunctioning. After the accelerometer recovers, the accelerometer driver can clear the data in that row (e.g., fill it with 0 or other values). After clearing, the data in the column storing the status identifier in that row will be restored to normal values.

[0109] In other embodiments, the state management array can also be a 2-row, multi-column array, with one column used to store the sensor identifier and state identifier of an accelerometer that is malfunctioning.

[0110] For example, the accelerometer driver can maintain a status flag for each accelerometer, with the value of each flag indicating the accelerometer's status. For instance, if an accelerometer's status flag is a first value, the accelerometer is functioning normally; if it is a second value, the accelerometer is malfunctioning. The first and second values ​​can be any values ​​that are different from each other.

[0111] It should be noted that in other embodiments, the accelerometer driver may also maintain status flags indicating the state (normal or abnormal) of each accelerometer in other ways, which is not limited here.

[0112] In some embodiments, the acceleration data collected by the accelerometer may include acceleration data from multiple axes (i.e., multiple directions corresponding one-to-one with the axes). The accelerometer driver can determine that the accelerometer is malfunctioning if, in the acceleration data collected by the accelerometer, the absolute value of the acceleration data for any axis is greater than a first acceleration threshold within a first time period, or if the absolute values ​​of the acceleration data for any axis for a consecutive first number of times are greater than a second acceleration threshold. The driver then configures the accelerometer's status flag to indicate that the accelerometer is malfunctioning (e.g., adding the accelerometer's sensor identifier to the status management array and configuring the sensor's status flag to a second value).

[0113] The knuckle detection driver is used to obtain acceleration data (e.g., IMU 01 data, IMU 02 data) collected by each acceleration sensor (e.g., IMU 01, IMU 02) of the mobile phone 10 and the status of each acceleration sensor from the acceleration sensor driver, and selects the acceleration data of the acceleration sensor without abnormality to report to the knuckle detection HAL according to the status of each acceleration sensor.

[0114] For example, if IMU 01 malfunctions, the knuckle detection driver can report IMU 02 data to the knuckle detection HAL. If IMU 02 malfunctions, the knuckle detection driver can report IMU 01 data to the knuckle detection HAL. If both IMU 01 and IMU 02 are functioning normally, the knuckle detection driver can report both IMU 01 and IMU 02 data to the knuckle detection HAL.

[0115] In some embodiments, the knuckle detection driver can use the Qualcomm Messaging Interface (QMI) to communicate with the knuckle detection HAL via event distribution management, sensor client manager, etc., to report acceleration data to the knuckle detection HAL.

[0116] The hardware layer may include hardware within the mobile phone 10, such as IMU 01 and IMU 02. IMU 01 and IMU 02 are used to collect acceleration data from the mobile phone 10. For example, the acceleration data may include acceleration data in three directions (three axes), respectively... Figure 1A The X direction, Y direction, and Z direction perpendicular to the X and Y directions are shown.

[0117] It should be noted that, Figure 4 The system architecture of the mobile phone 10 shown is only an example. In other embodiments, the software layer of the mobile phone 10 may include more or fewer layers, each layer may include more or fewer modules, and some layers / modules may be merged or split, which is not limited here.

[0118] Below, in conjunction with Figure 4 The system architecture shown illustrates the technical solution of this application.

[0119] First, we introduce the technical solution for maintaining the status of the accelerometer through accelerometer driving.

[0120] The accelerometer driver can maintain the state of each accelerometer in the mobile phone 10. The following is an example of the accelerometer driver maintaining the state of the first accelerometer (e.g., IMU 01 or IMU 02) to introduce the technical solution of the accelerometer driver maintaining the state of the accelerometer.

[0121] In some embodiments, the accelerometer driver can maintain the state of the accelerometer based on the relationship between one or more timing values ​​and count values ​​and corresponding threshold values ​​(timing threshold corresponding to timing value, count threshold corresponding to count value). The timing value indicates the duration for which the absolute value of acceleration data on at least one axis is continuously greater than or equal to an acceleration threshold during the accelerometer data acquisition process; the count value indicates the number of times the absolute value of acceleration data on at least one axis is continuously greater than or equal to an acceleration threshold during the accelerometer data acquisition process.

[0122] First, it is introduced that the accelerometer driver can maintain the state of the accelerometer based on the relationship between timing values ​​and count values ​​and corresponding threshold values ​​(timing threshold corresponding to timing value, counting threshold corresponding to count value).

[0123] For example, Figure 5A According to some embodiments of this application, a schematic flowchart of an accelerometer driver maintaining the state of an accelerometer sensor is shown, wherein the main body executing this flowchart is the accelerometer sensor driver. Figure 5A As shown, the method includes the following steps:

[0124] S501, acquire acceleration data from the first accelerometer.

[0125] The accelerometer driver can acquire acceleration data from a first accelerometer (e.g., IMU 01 or IMU 02). Each time the accelerometer driver acquires acceleration data, it includes acceleration data in three directions (three axes): X-axis acceleration data, Y-axis acceleration data, and Z-axis acceleration data.

[0126] S502, determine whether the following conditions are met: the absolute value of the X-axis acceleration data is less than the acceleration threshold, the absolute value of the Y-axis acceleration data is less than the acceleration threshold, and the absolute value of the Z-axis acceleration data is less than the acceleration threshold.

[0127] The accelerometer driver determines whether the absolute values ​​of the X-axis acceleration data, Y-axis acceleration data, and Z-axis acceleration data are all less than the acceleration threshold. If these conditions are met, it indicates that the acceleration data of the three axes of the first accelerometer are all within the normal range and there is no axis jamming, and proceeds to step S503; otherwise, if the acceleration data of one or more of the three axes is greater than or equal to the acceleration threshold, it indicates that one or more axes may be jamming, and proceeds to step S504 for further determination.

[0128] In some embodiments, the acceleration threshold can be referred to the description of the first acceleration threshold and the second acceleration threshold described above, and will not be repeated here.

[0129] S503, resets the timer and counter values.

[0130] The accelerometer driver resets the timing and count values ​​when the absolute values ​​of the acceleration data on all three axes are less than the acceleration threshold.

[0131] It should be noted that an accelerometer can collect acceleration data from multiple axes in a single data acquisition. If the absolute value of the acceleration data from at least one axis in a single acquisition is greater than or equal to the acceleration threshold, it is counted as one acquisition.

[0132] S504, determine whether the count value is less than the count threshold and the timing value is less than the timing threshold.

[0133] If the absolute value of the acceleration data for one or more of the three axes driven by the accelerometer is greater than or equal to the acceleration threshold, it can be determined whether the count value is less than the count threshold and whether the timing value is less than the timing threshold. If the count value is less than the count threshold and the timing value is less than the timing threshold, it indicates that the first accelerometer is not stuck, and the process proceeds to step S505. If the count value is greater than or equal to the count threshold and / or the timing value is greater than or equal to the timing threshold, it indicates that the first accelerometer may be stuck, and the process proceeds to step S506 for further determination.

[0134] S505 adds timing and counting values.

[0135] The accelerometer driver can increase the timing and count values ​​when the count value is less than the counting threshold and the timing value is less than the timing threshold.

[0136] S506 determines whether the count value is equal to the count threshold or whether the timing value is equal to the timing threshold.

[0137] The accelerometer driver determines whether the count value is equal to the count threshold or the timing value is equal to the timing threshold. If the count value is equal to the count threshold and / or the timing value is equal to the timing threshold, it indicates that the first accelerometer is stuck (an abnormality has occurred), and proceeds to step S508; otherwise, if the count value is not equal to the count threshold and the timing value is not equal to the timing threshold, it indicates that the first accelerometer is not stuck (normal), and proceeds to step S507.

[0138] S507 configures the status of the first accelerometer sensor to normal.

[0139] The accelerometer driver can configure the state of the first accelerometer to normal when the count value is not equal to the count threshold and the timing value is not equal to the timing threshold.

[0140] For example, the accelerometer driver can configure a status flag indicating the state of the first accelerometer to indicate that the first accelerometer is in a normal state. For instance, the accelerometer driver can configure the status flag of the first accelerometer to the aforementioned first value. As another example, the accelerometer driver can delete the sensor flag of the first accelerometer from the status manager array.

[0141] It should be noted that in some other embodiments, the acceleration sensor driver may also configure the state of the first acceleration sensor to normal in other ways, which is not limited here.

[0142] For example, in the case where the accelerometer driver indicates the state of the accelerometer by maintaining a state management data set, the accelerometer driver can configure the state of the first accelerometer to normal based on the following steps S507A to S507C.

[0143] S507A, traverse the state management array.

[0144] The accelerometer driver iterates through the state management array to obtain the sensor identifiers in the state management array.

[0145] S507B determines whether the sensor identifier encountered during traversal is the sensor identifier of the first accelerometer.

[0146] During the traversal of the state management array, the accelerometer driver can determine whether the sensor identifier encountered is the sensor identifier of the first accelerometer. If it is, it indicates that the state management array currently indicates that the first accelerometer is malfunctioning, and proceeds to step S507C; otherwise, it proceeds to step S507A to continue traversing until all data in the state management array has been traversed.

[0147] It should be noted that if no sensor identifier matching the sensor identifier of the first accelerometer is found after traversing the state management array, it means that the state management array indicates that the first accelerometer is currently functioning normally, and there is no need to maintain the state management array.

[0148] S507C, clear the sensor identifier and status identifier of the first accelerometer in the status management array.

[0149] When the accelerometer driver encounters a sensor identifier that matches the first accelerometer, it can clear both the sensor identifier and status identifier of the first accelerometer from the status management array. The cleared status identifier will then have a normal value.

[0150] S508 configures the state of the first accelerometer sensor as abnormal.

[0151] The accelerometer driver can configure the state of the first accelerometer to be abnormal if the count value equals the count threshold or the timing value equals the timing threshold.

[0152] For example, the accelerometer driver can configure a status flag indicating the state of the first accelerometer to indicate an abnormal state of the first accelerometer. For instance, the accelerometer driver can configure the status flag of the first accelerometer to the aforementioned second value. As another example, the accelerometer driver can add the sensor identifier of the first accelerometer to the status manager array and configure the corresponding status flag of the first accelerometer in the status management data to a third value (or leave it unchanged if the sensor identifier of the first accelerometer already exists in the status manager array).

[0153] It should be noted that in some other embodiments, the accelerometer driver may also configure the state of the first accelerometer to be abnormal in other ways, which is not limited here.

[0154] For example, in the case where the accelerometer driver indicates the state of the accelerometer by maintaining a state management data set, the accelerometer driver can configure the state of the first accelerometer to be abnormal based on the following steps S508A to S508F.

[0155] S508A, traverse the state manager array.

[0156] The accelerometer driver iterates through the state management array to obtain the sensor identifiers in the state management array.

[0157] S508B determines whether the sensor identifier of the first accelerometer sensor exists in the state management array.

[0158] The accelerometer driver can iterate through the state management array to determine whether the sensor identifier of the first accelerometer exists in the state management array. If the sensor identifier of the first accelerometer exists in the state management array, it means that the state management array currently indicates that the first accelerometer is abnormal, and no maintenance is required for the state management array; proceed to step S508C. Otherwise, if the sensor identifier of the first accelerometer does not exist in the state management array, it means that the state management array currently indicates that the first accelerometer is normal, and maintenance is required for the state management array; proceed to step S508D.

[0159] S508C, Exit traversal.

[0160] If the sensor identifier of the first accelerometer exists in the state management array, the accelerometer driver can exit the traversal.

[0161] S508D, traverse the state manager array.

[0162] If the sensor identifier of the first accelerometer does not exist in the state management array, the accelerometer driver can traverse the state manager array to find the storage location (e.g., a row, column, or element in the array) where the sensor identifier is not stored.

[0163] S508E determines whether the status flag of the traversed storage location is a normal value.

[0164] During the traversal of the state management array, the accelerometer driver can determine whether the state flag of the traversed storage location is a normal value (indicating that the storage location does not store the state flag and sensor flag of the accelerometer). If the state flag of the traversed storage location is a normal value, it means that the sensor flag and state flag of the first accelerometer can be stored in that storage location, and proceed to step S508F; otherwise, it means that the sensor flag and state flag of the first accelerometer cannot be stored in that storage location, and proceed to step S508D to continue traversing the next storage location.

[0165] S508F adds the sensor identifier and status identifier of the first accelerometer to the traversed storage location.

[0166] If the status flag of the traversed storage location is a normal value, the accelerometer driver can add the sensor flag and status flag of the first accelerometer (with the value of the aforementioned third value) to the traversed storage location.

[0167] Based on the above method, the accelerometer driver can maintain the state of each accelerometer in the mobile phone 10, so that other modules can obtain the state of the accelerometer from the accelerometer driver and implement related services based on the obtained state of the accelerometer.

[0168] The following describes how the accelerometer driver maintains the state of the accelerometer based on the relationship between timing values ​​and timing thresholds.

[0169] For example, Figure 5B According to some embodiments of this application, a schematic diagram of another process for maintaining the state of an accelerometer sensor is shown, in which the accelerometer sensor driver is the main entity executing the process. Figure 5B As shown, the method includes the following steps:

[0170] S501′, acquire acceleration data from the first accelerometer.

[0171] S502′, determine whether the following conditions are met: the absolute value of the X-axis acceleration data is less than the acceleration threshold, the absolute value of the Y-axis acceleration data is less than the acceleration threshold, and the absolute value of the Z-axis acceleration data is less than the acceleration threshold.

[0172] It should be noted that steps S501′ to S502′ are essentially the same as steps S501 to S502, and will not be described in detail here.

[0173] S503′, Reset timer value.

[0174] The accelerometer driver resets the timing value when the absolute value of the acceleration data on all three axes is less than the acceleration threshold.

[0175] S504′ determines whether the timing value is less than the timing threshold.

[0176] If the absolute value of the acceleration data for one or more of the three axes driven by the accelerometer is greater than or equal to an acceleration threshold, it can be determined whether the timing value is less than a timing threshold. If the timing value is less than the timing threshold, it indicates that the first accelerometer is not stuck, and the process proceeds to step S505'. If the timing value is greater than or equal to the timing threshold, it indicates that the first accelerometer may be stuck, and the process proceeds to step S506' for further determination.

[0177] S505′, adds a timing threshold.

[0178] The accelerometer sensor driver can increase the timing value if the timing value is lower than the timing threshold.

[0179] S506′ determines whether the timing value is equal to the timing threshold.

[0180] The accelerometer driver determines whether the timing value is equal to the timing threshold. If the timing value is equal to the timing threshold, it indicates that the first accelerometer is stuck (an abnormality has occurred), and proceeds to step S508'; otherwise, if the timing value is not equal to the timing threshold, it indicates that the first accelerometer is not stuck (normal), and proceeds to step S507'.

[0181] S507′, configure the status of the first accelerometer sensor to normal.

[0182] The accelerometer driver can configure the state of the first accelerometer to normal even if the timing value is not equal to the timing threshold. See step S507 for details, which will not be elaborated here.

[0183] S507A′, traverse the state management array.

[0184] S507B′ determines whether the sensor identifier encountered during traversal is the sensor identifier of the first accelerometer.

[0185] S507C′, clear the sensor identifier and status identifier of the first accelerometer in the status management array.

[0186] It should be noted that steps S507A′ to S507C′ are essentially the same as steps S507A to S507C, and will not be described in detail here.

[0187] S508′, configure the state of the first accelerometer sensor as abnormal.

[0188] The accelerometer driver can configure the state of the first accelerometer to be abnormal when the timing value equals the timing threshold. See step S508 for details, which will not be elaborated here.

[0189] S508A′, traverse the state manager array.

[0190] S508B′, determine whether the sensor identifier of the first accelerometer sensor exists in the state management array.

[0191] S508C', Exit traversal.

[0192] S508D′, traverse the state manager array.

[0193] S508E′ determines whether the status flag of the traversed storage location is a normal value.

[0194] S508F′ adds the sensor identifier and status identifier of the first accelerometer to the traversed storage location.

[0195] It should be noted that steps S508A′ to S508F′ are essentially the same as steps S508A to S508F, and will not be described in detail here.

[0196] Based on the above method, the accelerometer driver can maintain the state of each accelerometer in the mobile phone 10, so that other modules can obtain the state of the accelerometer from the accelerometer driver and implement related services based on the obtained state of the accelerometer.

[0197] The following describes how the accelerometer driver maintains the state of the accelerometer based on the relationship between the count value and the count threshold.

[0198] For example, Figure 5C According to some embodiments of this application, a schematic flowchart of another accelerometer sensor driver maintaining the state of an accelerometer sensor is shown, wherein the accelerometer sensor driver is the main entity executing this process. Figure 5C As shown, the method includes the following steps:

[0199] S501″, acquire acceleration data from the first accelerometer.

[0200] S502″, determine whether the following conditions are met: the absolute value of the X-axis acceleration data is less than the acceleration threshold, the absolute value of the Y-axis acceleration data is less than the acceleration threshold, and the absolute value of the Z-axis acceleration data is less than the acceleration threshold.

[0201] It should be noted that steps S501″ to S502″ are essentially the same as steps S501 to S502, and will not be described in detail here.

[0202] S503″, Reset the counter value.

[0203] The accelerometer driver resets the count value when the absolute value of the acceleration data on all three axes is less than the acceleration threshold.

[0204] S504″ determines whether the count value is less than the count threshold.

[0205] If the absolute value of the acceleration data for one or more of the three axes driven by the accelerometer is greater than or equal to the acceleration threshold, it can be determined whether the count value is less than the counting threshold. If the count value is less than the counting threshold, it indicates that the first accelerometer is not stuck, and proceed to step S505″. ​​If the count value is greater than or equal to the counting threshold, it indicates that the first accelerometer may be stuck, and proceed to step S506″ for further determination.

[0206] S505″, adds a counting threshold.

[0207] The accelerometer driver can increase the count value if the count value is less than the count threshold.

[0208] S506″ determines whether the count value is equal to the count threshold.

[0209] The accelerometer driver determines whether the count value is equal to the count threshold. If the count value is equal to the count threshold, it indicates that the first accelerometer is stuck (an abnormality has occurred), and proceeds to step S508″; otherwise, if the count value is not equal to the count threshold, it indicates that the first accelerometer is not stuck (normal), and proceeds to step S507″.

[0210] S507″ configures the status of the first accelerometer sensor to normal.

[0211] The accelerometer driver can configure the state of the first accelerometer to normal even if the count value is not equal to the count threshold. See step S507 for details, which will not be elaborated here.

[0212] S507A″, traverse the state management array.

[0213] S507B″ determines whether the sensor identifier encountered during traversal is the sensor identifier of the first accelerometer.

[0214] S507C″ clears the sensor identifier and status identifier of the first accelerometer in the status management array.

[0215] It should be noted that steps S507A″ to S507C″ are essentially the same as steps S507A to S507C, and will not be described in detail here.

[0216] S508″ configures the state of the first accelerometer sensor as abnormal.

[0217] The accelerometer driver can configure the state of the first accelerometer to be abnormal when the count value equals the count threshold. See step S508 for details, which will not be elaborated here.

[0218] S508A″, traverse the state manager array.

[0219] S508B″ determines whether the sensor identifier of the first accelerometer sensor exists in the state management array.

[0220] S508C″, Exit traversal.

[0221] S508D″, traverse the state manager array.

[0222] S508E″ determines whether the status flag of the traversed storage location is a normal value.

[0223] S508F″ adds the sensor identifier and status identifier of the first accelerometer to the traversed storage location.

[0224] It should be noted that steps S508A″ to S508F″ are essentially the same as steps S508A to S508F, and will not be described in detail here.

[0225] Based on the above method, the accelerometer driver can maintain the state of each accelerometer in the mobile phone 10, so that other modules can obtain the state of the accelerometer from the accelerometer driver and implement related services based on the obtained state of the accelerometer.

[0226] The following describes a technical solution for the joint detection driver to report acceleration data to the joint detection HAL based on the state of the accelerometer.

[0227] For example, Figure 6 According to some embodiments of this application, a schematic diagram of a process is shown in which a knuckle detection driver reports acceleration data to the knuckle detection HAL based on the state of an accelerometer sensor. Figure 6 As shown, the main execution entity of this process is the knuckle detection driver, and the process includes the following steps:

[0228] S601A, acquire acceleration data from IMU 01.

[0229] The knuckle detection driver can acquire acceleration data from IMU 01 via the accelerometer driver. For example, the knuckle detection driver can buffer the acceleration data acquired by IMU 01 during the acquisition of acceleration data by the accelerometer driver.

[0230] S601B, acquire acceleration data from IMU 02.

[0231] The knuckle detection driver can acquire acceleration data from IMU 02 via the accelerometer driver. For example, the knuckle detection driver can buffer the acceleration data acquired by IMU 02 during the acquisition process by the accelerometer driver.

[0232] It should be noted that the execution order of steps S601A and S601B is not limited in the embodiments of this application. Steps S601A and S601B can be executed in parallel, or step S601A can be executed first and then step S601B can be executed, or step S601B can be executed first and then step S601A can be executed.

[0233] S602 reports acceleration data based on the status of IMU 01 and IMU 02.

[0234] After acquiring the acceleration data of IMU 01 and IMU 02, the knuckle detection driver can report the acceleration data to the knuckle detection HAL based on the status of IMU 01 and IMU 02.

[0235] For example, in the event of an IMU 01 malfunction, the knuckle detection driver can report the acceleration data of IMU 02 to the knuckle detection HAL. In the event of an IMU 02 malfunction, the knuckle detection driver can report the acceleration data of IMU 01 to the knuckle detection HAL. If both IMU 01 and IMU 02 are malfunctioning or both are functioning normally, if the user's operation is located in display area 11A or the non-flexible screen 12, the knuckle detection driver can report the acceleration data of IMU 01 to the knuckle detection HAL; if the user's operation is located in display area 1B, the knuckle detection driver can report the acceleration data of IMU 02 to the knuckle detection HAL.

[0236] In other embodiments, if both IMU 01 and IMU 02 are malfunctioning, the knuckle detection driver can also report the acceleration data of both IMU 01 and IMU 02 to the knuckle detection HAL.

[0237] For example, the knuckle detection driver can report acceleration data to the knuckle detection HAL based on the states of IMU 01 and IMU 02 according to the following steps S602A to S602E.

[0238] S602A, determine if IMU 01 is abnormal.

[0239] The knuckle detection driver can determine whether IMU 01 is abnormal. If IMU 01 is abnormal, it means that the acceleration data collected by IMU 01 may be inaccurate, and proceed to step S602B; otherwise, proceed to step S602C.

[0240] In some embodiments, the knuckle detection driver obtains the status identifier or status management array of IMU 01 from the acceleration sensor driver. If the status identifier of IMU 01 is the aforementioned first value or there is no sensor identifier of IMU 01 in the status management array, it is determined that IMU 01 is normal; otherwise, if the status identifier of IMU 01 is the aforementioned second value or there is a sensor identifier of IMU 01 in the status management array, it is determined that IMU 01 is abnormal.

[0241] S602B reports acceleration data from IMU 02.

[0242] The knuckle detection driver can report the acceleration data of IMU 02 to the knuckle detection HAL in the event of an IMU 01 malfunction.

[0243] In some embodiments, the knuckle detection driver can report two sets of acceleration data to the knuckle detection HAL. In the event of an IMU 01 malfunction, the two sets of data are identical, both being acceleration data from IMU 02.

[0244] S602C, determine if IMU 02 is abnormal.

[0245] The knuckle detection driver can determine whether IMU 02 is abnormal. If IMU 02 is abnormal, it means that the acceleration data collected by IMU 02 may be inaccurate, and proceed to step S602D; otherwise, proceed to step S602E.

[0246] In some embodiments, the knuckle detection driver obtains the status identifier or status management array of IMU 02 from the acceleration sensor driver. If the status identifier of IMU 02 is the aforementioned first value or there is no sensor identifier of IMU 02 in the status management array, it is determined that IMU 02 is normal; otherwise, if the status identifier of IMU 02 is the aforementioned second value or there is a sensor identifier of IMU 02 in the status management array, it is determined that IMU 02 is abnormal.

[0247] S602D reports acceleration data from IMU 01.

[0248] The knuckle detection driver can report the acceleration data of IMU 01 to the knuckle detection HAL when IMU 01 is normal and IMU 02 is abnormal.

[0249] In some embodiments, the knuckle detection driver can report two sets of acceleration data to the knuckle detection HAL. In the event of an IMU 02 malfunction, the two sets of data are identical, both being acceleration data from IMU 01.

[0250] S602E reports acceleration data from IMU 01 and IMU 02.

[0251] The knuckle detection driver can report the acceleration data of IMU 01 and IMU 02 to the knuckle detection HAL when both IMU 01 and IMU 02 are functioning normally.

[0252] In some embodiments, the knuckle detection driver can report two sets of acceleration data to the knuckle detection HAL. When both IMU 01 and IMU 02 are functioning normally, the two sets of data are the acceleration data of IMU 01 and the acceleration data of IMU 02, respectively.

[0253] In some embodiments, when both IMU 01 and IMU 02 are functioning normally, if the user's operation is located in the display area 11A or the non-flexible screen 12, the knuckle detection HAL can identify whether the user's operation is a knuckle operation based on the acceleration data of IMU 01; if the user's operation is located in the display area 11B, the knuckle detection HAL can identify whether the user's operation is a knuckle operation based on the acceleration data of IMU 02.

[0254] Based on the above method, the knuckle detection driver can select the acceleration data collected by the IMU that does not show any abnormalities in IMU 01 and IMU 02 and report it to the knuckle detection HAL. This allows the knuckle detection HAL to identify knuckle operations based on more accurate data, which helps to improve the accuracy of the mobile phone 10 in recognizing knuckle operations.

[0255] The following is combined Figures 3 to 7 This paper introduces the technical solution of this application.

[0256] For example, Figure 7 According to some embodiments of this application, an interactive flowchart of an operation detection method is shown.

[0257] like Figure 7 As shown, the method includes the following steps:

[0258] S701, the knuckle application registers with the knuckle detection service to monitor knuckle detection.

[0259] When the knuckle app is running, or when the user enables the knuckle operation-related functions of the knuckle app or the operating system of the phone 10, the knuckle app registers with the knuckle detection service to listen for knuckle detection.

[0260] It should be noted that after the knuckle application registers with the knuckle detection service to listen for knuckle detection, it can receive the knuckle operation recognition results sent by the knuckle detection service and perform related business based on the knuckle operation recognition results.

[0261] S702, the knuckle detection service sends a detection activation request to the knuckle detection HAL.

[0262] In response to the registration listener of the knuckle application, the knuckle detection service sends a knuckle detection activation request to the knuckle detection HAL to activate the knuckle detection HAL to instruct the knuckle detection driver to cache the acceleration data collected by the accelerometer.

[0263] S703, HAL activation for knuckle detection.

[0264] In response to the knuckle detection activation request, the knuckle detection HAL can activate the knuckle detection driver detection function of the knuckle detection driver, so that the knuckle detection driver caches the acceleration data collected by the accelerometer.

[0265] S704, the knuckle detection driver caches acceleration data from each accelerometer.

[0266] After activating knuckle detection, the knuckle detection driver can cache acceleration data from various accelerometers. For example, it can cache acceleration data collected by IMU 01 and acceleration data collected by IMU 02.

[0267] S705, in response to a user action, the knuckle detection service sends a knuckle detection request to the knuckle detection service.

[0268] The knuckle detection service can respond to user actions by sending a knuckle detection request to the knuckle detection service.

[0269] For example, when a mobile phone 10 (e.g., IMS) detects a user action (e.g., a down event), it can distribute a down event to the knuckle detection service. In response to the down event, the knuckle detection service can then send a knuckle detection request to the knuckle detection service.

[0270] It should be noted that in some other embodiments, the knuckle detection service may also send a knuckle detection request to the knuckle detection service in other circumstances, which is not limited here.

[0271] S706, the knuckle detection HAL sends a data acquisition request to the knuckle detection driver.

[0272] The knuckle detection HAL can respond to knuckle detection requests by sending a data acquisition request to the knuckle detection driver, instructing the knuckle detection driver to report acceleration data.

[0273] It should be noted that after receiving a data acquisition request, the knuckle detection driver can report the acceleration data of IMU 01 and / or IMU 02 to the knuckle detection HAL based on the status of IMU 01 and IMU 02.

[0274] S707, the knuckle detection driver determines the status of IMU 01 and IMU 02.

[0275] In response to a data acquisition request, the knuckle detection driver can acquire and determine the status (normal or abnormal) of IMU01 and IMU02 from the accelerometer driver, so as to report the acceleration data of IMU01 and / or IMU02 to the knuckle detection HAL.

[0276] For example, the knuckle detection driver can first determine whether IMU 01 is abnormal. If IMU 01 is abnormal, proceed to step S708A; if IMU 01 is normal, then determine whether IMU 02 is abnormal. If IMU 02 is abnormal, proceed to step S708C; otherwise, if IMU 02 is normal, proceed to step S708B. For details, please refer to the aforementioned steps S602, S602A to S602C, which will not be elaborated upon here.

[0277] In the event of an IMU 01 malfunction, the S708A's knuckle detection driver reports the acceleration data of IMU 02 to the knuckle detection HAL.

[0278] In the event of an IMU 01 malfunction, the knuckle detection driver can report the acceleration data of IMU 02 to the knuckle detection HAL. For details, please refer to the aforementioned 602B; further explanation is omitted here.

[0279] When both IMU 01 and IMU 02 are functioning normally, the S708B knuckle detection driver reports the acceleration data of IMU 01 and IMU 02 to the knuckle detection HAL.

[0280] With both IMU 01 and IMU 02 functioning normally, the knuckle detection driver can report the acceleration data of IMU 01 and IMU 02 to the knuckle detection HAL. For details, please refer to the aforementioned 602E; further explanation is omitted here.

[0281] In the event of an IMU 02 malfunction, the S708C's knuckle detection driver reports the acceleration data of IMU 01 to the knuckle detection HAL.

[0282] In the event of an IMU 02 malfunction, the knuckle detection driver can report the acceleration data of IMU 01 to the knuckle detection HAL. For details, please refer to the aforementioned 602D; further explanation is omitted here.

[0283] S709, the knuckle detection HAL calls the knuckle algorithm to identify knuckle operations.

[0284] After receiving acceleration data reported by the knuckle detection driver, the knuckle detection HAL can use this acceleration data to call the knuckle algorithm to identify knuckle operations and obtain the knuckle recognition result. This knuckle recognition result can indicate whether a knuckle operation was detected and / or the type of knuckle operation (e.g., knuckle click, knuckle double click, multiple knuckle taps, knuckle slide, etc.).

[0285] In some embodiments, in the event of an IMU 01 malfunction, the knuckle detection HAL can identify whether a user operation is a knuckle operation by inputting acceleration data from the IMU 02 into the knuckle algorithm.

[0286] In some embodiments, in the event of an IMU 02 malfunction, the knuckle detection HAL can identify whether a user operation is a knuckle operation by inputting acceleration data from the IMU 01 into the knuckle algorithm.

[0287] In some embodiments, when both IMU 01 and IMU 02 are functioning normally, if the user's operation is located in display area 11A or non-flexible screen 12, the knuckle detection HAL can input the acceleration data of IMU 01 into the knuckle algorithm to identify whether the user's operation is a knuckle operation; if the user's operation is located in display area 11B, the knuckle detection HAL can input the acceleration data of IMU 02 into the knuckle algorithm to identify whether the user's operation is a knuckle operation.

[0288] S710, the knuckle detection HAL sends knuckle recognition results to the knuckle application through the knuckle detection service.

[0289] After obtaining the knuckle recognition results, the knuckle detection HAL can send the knuckle recognition results to knuckle applications that have registered to listen for knuckle detection through the knuckle detection service.

[0290] S711, the knuckle application performs related business based on the knuckle recognition results.

[0291] After receiving the knuckle recognition result, the knuckle application can perform related tasks (such as screenshotting, screen recording, etc.) according to the type of knuckle operation.

[0292] Based on the above method, when IMU 01 is malfunctioning, mobile phone 10 can use the acceleration data collected by IMU 02 to identify whether the first operation is a knuckle operation, and when IMU 02 is malfunctioning, it can use the acceleration data collected by IMU 01 to identify whether the first operation is a knuckle operation. In this way, the accuracy of mobile phone 10 in recognizing knuckle operations can be improved.

[0293] It should be noted that the above description of the embodiments of this application using an IMU is only an example. In other embodiments, the IMU can be replaced with any other accelerometer, which is not limited here.

[0294] It should be noted that the above description of the embodiments of this application using mobile phone 10 is only an example. In other embodiments, mobile phone 10 can also be replaced by other electronic devices that include multiple acceleration sensors, which is not limited here.

[0295] This application also provides a service processing method. An electronic device can collect acceleration data through multiple acceleration sensors, and determine whether each acceleration sensor is abnormal (e.g., whether there is a jammed axis) based on the acceleration data collected by each acceleration sensor. Then, the electronic device can use the acceleration data collected by one or more acceleration sensors that are not abnormal to perform related services, such as identifying whether the user's click operation is a knuckle operation, rotating the display direction of the electronic device, and recording the motion information of the electronic device.

[0296] For example, Figure 8 According to some embodiments of this application, a flowchart of another operation detection method is shown. The method is executed by an electronic device, such as... Figure 8 As shown, the method includes the following steps:

[0297] S801 acquires acceleration data collected by multiple acceleration sensors of an electronic device.

[0298] Applications, services, or other modules in an electronic device can acquire acceleration data collected by multiple accelerometers in the electronic device.

[0299] S802, based on the status of each accelerometer, uses acceleration data collected by one or more accelerometers that have not shown any abnormalities to implement related services.

[0300] After acquiring acceleration data from multiple accelerometers, an electronic device can perform relevant services based on the status (normal or abnormal) of each accelerometer, using acceleration data from one or more accelerometers that are not abnormal (normal or not abnormal). Examples include services such as detecting whether a user operation is a critical operation, adjusting the display orientation of the device, and recording the motion information of the electronic device.

[0301] The methods by which electronic devices acquire the states of each accelerometer sensor can be referred to the foregoing. Figures 4 to 7 The specific implementation examples are not described in detail here.

[0302] Based on this, electronic devices can use accelerometers that have not malfunctioned to perform related services, thereby improving the accuracy of these services.

[0303] In other embodiments, the electronic device may first determine the state of each acceleration sensor and acquire acceleration data collected by one or more acceleration sensors that have not shown any abnormalities, and then implement related services based on the acquired acceleration data.

[0304] Based on the foregoing embodiments, this application also provides an operation detection method applied to an electronic device, the electronic device including a plurality of acceleration sensors, the plurality of acceleration sensors including at least a first acceleration sensor and a second acceleration sensor.

[0305] For example, Figure 9 According to some embodiments of this application, a flowchart of another operation detection method is shown, wherein the execution subject of the method is an electronic device. Figure 9 As shown, the method includes the following steps:

[0306] S901 detected the user's first action.

[0307] When an electronic device detects a user's first action, it triggers the method provided in the embodiments of this application.

[0308] For example, the first operation can be a touch operation such as a user clicking, sliding, or pinching on any display area (or position) of any screen on an electronic device using their fingers, knuckles, or other body parts or objects (such as a stylus).

[0309] S902A, when the first accelerometer is normal and the second accelerometer is abnormal, uses the first acceleration data collected by the first accelerometer to identify whether the first operation is a knuckle operation.

[0310] After detecting the user's first operation, the electronic device can obtain the status (normal or abnormal) of the first acceleration sensor and the second acceleration sensor. If the first acceleration sensor is normal and the second acceleration sensor is abnormal, the first acceleration data collected by the first acceleration sensor can be used to identify whether the first operation is a knuckle operation.

[0311] The methods by which electronic devices acquire the states of each accelerometer sensor can be referred to the foregoing. Figures 4 to 7 The specific implementation examples are not described in detail here.

[0312] S902B, when the first accelerometer is malfunctioning and the second accelerometer is functioning normally, uses the second acceleration data collected by the second accelerometer to identify whether the first operation is a knuckle operation.

[0313] After detecting the user's first operation, the electronic device can obtain the status (normal or abnormal) of the first acceleration sensor and the second acceleration sensor. If the first acceleration sensor is abnormal and the second acceleration sensor is normal, the electronic device can use the second acceleration data collected by the second acceleration sensor to identify whether the first operation is a knuckle operation.

[0314] The methods by which electronic devices acquire the states of each accelerometer sensor can be referred to the foregoing. Figures 4 to 7 The specific implementation examples are not described in detail here.

[0315] Based on this, electronic devices can use acceleration data collected by accelerometers that have not shown any abnormalities to identify whether the first operation is a knuckle operation, which can improve the accuracy of knuckle operation identification.

[0316] further, Figure 10 According to some embodiments of this application, a structural schematic diagram of a mobile phone 10 is shown.

[0317] like Figure 10As shown, the mobile phone 10 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identity module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, one or more (N, where N is a positive integer) gyroscope sensors 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, one or more (N, where N is a positive integer) accelerometers 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0318] It should be noted that the phone 10 can have multiple accelerometer sensors. These multiple accelerometer sensors can be the same or different. For example, they can all be gyroscope sensor 180B, or they can be accelerometer 180E, or they can be other types of acceleration sensors.

[0319] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the mobile phone 10. In other embodiments of this application, the mobile phone 10 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0320] Processor 110 may include one or more processing units, such as application processors (APs), modem processors, graphics processing units (GPUs), image signal processors (ISPs), controllers, video codecs, digital signal processors (DSPs), baseband processors, and / or neural network processing units (NPUs). These different processing units may be independent devices or integrated into one or more processors.

[0321] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the aforementioned memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0322] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface, and / or a universal serial bus (USB) interface, etc.

[0323] USB port 130 is a USB standard compliant interface, specifically a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 130 can be used to connect a charger to charge the phone 10, or to transfer data between the phone 10 and peripheral devices. It can also be used to connect headphones for audio playback.

[0324] The charging management module 140 receives charging input from a charger, which can be a wireless charger or a wired charger. While charging the battery 142, the charging management module 140 can also supply power to the electronic device via the power management module 141.

[0325] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. In some embodiments, the power management module 141 and the charging management module 140 may also be housed in the same device.

[0326] The wireless communication function of mobile phone 10 can be realized through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

[0327] Antennas 1 and 2 are used to transmit and receive electromagnetic wave signals. Each antenna in mobile phone 10 can be used to cover one or more communication frequency bands. Different antennas can also be reused to improve antenna utilization. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0328] The mobile communication module 150 can provide solutions for wireless communication applications including 2G / 3G / 4G / 5G on the mobile phone 10. The mobile communication module 150 may include at least one filter, switch, power amplifier, low-noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via the antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to the modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

[0329] The wireless communication module 160 can provide solutions for wireless communication applications on the mobile phone 10, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signals, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification, and convert them into electromagnetic waves for radiation via antenna 2.

[0330] In some embodiments, antenna 1 of mobile phone 10 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, enabling mobile phone 10 to communicate with networks and other devices via wireless communication technology. The aforementioned wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time-Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The aforementioned GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0331] The mobile phone 10 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0332] Display screen 194 is used to display images, videos, etc.

[0333] The phone 10 can achieve shooting functions through the ISP, camera 193, video codec, GPU, display 194 and application processor.

[0334] Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element through the lens to generate an optical image. In some embodiments, mobile phone 10 may include one or N cameras 193, where N is a positive integer greater than 1.

[0335] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the mobile phone 10. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0336] Internal memory 121 can be used to store computer executable program code, including instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, applications required for at least one function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of mobile phone 10 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 110 executes various functional applications and data processing of mobile phone 10 by running instructions stored in internal memory 121 and / or instructions stored in memory located in the processor.

[0337] The phone 10 can perform audio functions, such as music playback and recording, through an audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0338] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0339] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The phone 10 can listen to music or make hands-free calls through the speaker 170A.

[0340] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the mobile phone 10 receives a call or voice message, the receiver 170B can be brought close to the user's ear to listen to the voice.

[0341] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C.

[0342] The 170D headphone jack is used to connect wired headphones.

[0343] Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes. Mobile phone 10 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, mobile phone 10 detects the intensity of the touch operation based on pressure sensor 180A. Mobile phone 10 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands.

[0344] The gyroscope sensor 180B can be used to determine the motion posture of the mobile phone 10. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the mobile phone 10 around three axes (i.e., the x, y, and z axes). The magnitude of this acceleration can be used to implement related services, such as detecting whether the user's operation is a knuckle operation, adjusting the display orientation of the mobile phone 10, recording the motion information of the mobile phone 10, etc.

[0345] The accelerometer 180E can detect the magnitude of the acceleration of the mobile phone 10 in various directions (generally three axes). The magnitude of this acceleration can be used to realize related functions, such as detecting whether the user operation is a knuckle operation, adjusting the display orientation of the mobile phone 10, recording the motion information of the mobile phone 10, etc.

[0346] The ambient light sensor 180L is used to detect ambient light levels. The phone 10 can adaptively adjust the display brightness 194 based on the detected ambient light level. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking photos. The ambient light sensor 180L can also work in conjunction with the proximity sensor 180G to detect whether the phone 10 is in a pocket, preventing accidental touches.

[0347] Touch sensor 180K, also known as a "touch device," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations (such as finger operations, knuckle operations, etc.) applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of mobile phone 10, in a different position than display screen 194.

[0348] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch buttons. Mobile phone 10 can receive button input and generate key signal inputs related to user settings and function control of mobile phone 10.

[0349] Motor 191 can generate vibration alerts.

[0350] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0351] The SIM card interface 195 is used to connect the SIM card.

[0352] It should be noted that the electronic device to which this application embodiment applies may adopt the same or similar structure as mobile phone 10, or may adopt a different structure than mobile phone 10, which is hereby limited.

[0353] This application also provides a computer-readable storage medium storing one or more program instructions that, when executed on an electronic device, cause the electronic device to perform the operation detection methods provided in the foregoing embodiments.

[0354] This application also provides a computer program product that, when run on an electronic device, causes the electronic device to execute the operation detection methods provided in the foregoing embodiments.

[0355] It should be noted that the terminology used in the implementation section of the embodiments of this application is only used to explain the specific embodiments of this application and is not intended to limit this application. In the description of the embodiments of this application, unless otherwise stated, " / " means "or", for example, A / B can mean A or B; "and / or" in this document is merely a description of the association relationship of related obstacles, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. In addition, in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, "at least one" or "one or more" means one, two or more.

[0356] It should be noted that in the embodiments of this application, "greater than or equal to" and the corresponding "less than", "equal to" can also be used together with "less than". For example, indicating that a certain parameter corresponds to case B when it is greater than or equal to A and case C when it is less than A can also be understood as the parameter corresponding to case B when it is greater than A and case C when it is less than or equal to A.

[0357] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature.

[0358] References to "one embodiment" or "some embodiments" as used in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0359] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially as a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium, or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0360] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the above method embodiments.

[0361] The above description is merely a specific implementation of the embodiments of this application, but the protection scope of the embodiments of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the embodiments of this application should be covered within the protection scope of the embodiments of this application. Therefore, the protection scope of the embodiments of this application should be determined by the protection scope of the claims.

Claims

1. An operation detection method, applied to electronic equipment, characterized in that, The electronic device is a foldable electronic device, comprising a first display screen and a second display screen, which are respectively disposed on opposite sides of the electronic device. The first display screen is a non-flexible screen, and the second display screen is a flexible screen. The electronic device includes a first accelerometer and a second accelerometer, with the first accelerometer disposed on one side of the folding axis and the second accelerometer disposed on the other side of the folding axis. The first accelerometer and the first display screen are located on the same side of the folding axis. Furthermore, the method includes: A first user action is detected, which is an action performed on the first display screen; When the first accelerometer is functioning normally, the first acceleration data collected by the first accelerometer is used to identify whether the first operation is a knuckle operation; If the first acceleration sensor is malfunctioning and the second acceleration sensor is functioning normally, the second acceleration data collected by the second acceleration sensor is used to identify whether the first operation is a knuckle operation.

2. The method according to claim 1, characterized in that, The anomalies include: In the acceleration data collected by the accelerometer in one or more directions, the absolute value of the acceleration data in at least one direction is greater than a first threshold for a continuous first time period. Alternatively, in one or more directions of acceleration data collected by the accelerometer, the absolute value of a first number of consecutive acceleration data points in at least one direction is greater than the second threshold.

3. The method according to claim 1, characterized in that, The method further includes: If the first operation is identified as the knuckle operation, the service corresponding to the knuckle operation is executed.

4. The method according to claim 1, characterized in that, Also includes: If both the first and second acceleration sensors are malfunctioning, the system identifies whether the first operation is a knuckle operation based on the acceleration data collected by the sensor closest to the operation position of the first operation.

5. The method according to claim 1, characterized in that, The method further includes: If the first operation is an operation in the screen area on one side of the second display screen and the first acceleration sensor is normal, the system identifies whether the first operation is a knuckle operation based on the first acceleration data. Alternatively, if the first operation is an operation in the screen area on the other side of the second display screen and the second accelerometer is functioning normally, the first operation can be identified as a knuckle operation based on the second acceleration data.

6. The method according to any one of claims 1 to 5, characterized in that, The step of identifying whether the first operation is a knuckle operation using the second acceleration data collected by the second acceleration sensor when the first acceleration sensor is malfunctioning and the second acceleration sensor is normal includes: Obtain the first state identifier of the first accelerometer and the second state identifier of the second accelerometer; When the first status indicator indicates that the first acceleration sensor is malfunctioning and the second status indicator indicates that the second acceleration sensor is functioning normally, the second acceleration data is used to identify whether the first operation is a knuckle operation.

7. The method according to claim 6, characterized in that, The electronic device includes an accelerometer driver, and the first status identifier and the second status identifier are maintained by the accelerometer driver.

8. The method according to claim 7, characterized in that, The acceleration sensor driver maintains the first state identifier and the second state identifier in the following manner: When the accelerometer driver detects the abnormality in the first accelerometer, it configures the first status flag to indicate that the first accelerometer is abnormal. When the accelerometer driver detects the abnormality in the second accelerometer, it configures the second status flag to indicate that the second accelerometer is abnormal. When the accelerometer driver detects that the first accelerometer does not exhibit the aforementioned abnormality, it configures the first status flag to indicate that the first accelerometer is functioning normally. If the accelerometer driver detects that the second accelerometer does not exhibit the aforementioned abnormality, it will configure the second status flag to indicate that the second accelerometer is functioning normally.

9. The method according to claim 7, characterized in that, The electronic device includes a knuckle detection driver and a knuckle detection hardware abstraction layer; and, when the first status indicator indicates that the first accelerometer is malfunctioning and the second status indicator indicates that the second accelerometer is functioning normally, the step of using the second acceleration data to identify whether the first operation is a knuckle operation includes: When the first status indicator indicates that the first accelerometer is malfunctioning and the second status indicator indicates that the second accelerometer is functioning normally, the knuckle detection driver sends the second acceleration data to the knuckle detection hardware abstraction layer. The knuckle detection hardware abstraction layer uses the second acceleration data to identify whether the first operation is a knuckle operation.

10. The method according to claim 1, characterized in that, The electronic device further includes at least one third acceleration sensor; and the method further includes: In the event of an anomaly in the first acceleration sensor and the second acceleration sensor, the third acceleration data collected by the at least one third acceleration sensor is used to identify whether the first operation is a knuckle operation.

11. A readable storage medium, characterized in that, The readable storage medium stores one or more programs that, when run on an electronic device, cause the electronic device to perform the method of any one of claims 1 to 10.

12. An electronic device, characterized in that, include: One or more processors, one or more memories, and multiple accelerometers; The one or more memories store one or more computer programs, which, when executed by the one or more processors, cause the electronic device to perform the method of any one of claims 1 to 10.

Images